Data Analytics: Should We Build Iron Man or R2D2?

Earlier this year, Alex Handy wrote an intriguing article on exploring the future of data analysis, which In this article Handy compared and contrasted two approaches to understanding the ever-increasing stream of data. One approach depends upon building “exoskeletal systems”, which enhance human comprehension. Hardy draws connections to this solution and “Iron Man”. The other approach would depend chiefly on autonomous robots or automated systems. This alternative, Hardy suggests, is more like “R2D2” from Star Wars. Ultimately, Handy concludes that “[d]evelopers should build Iron Man, not R2D2.”

Here at Digital Reasoning, we have been dealing with the challenges of automated understanding of massive amounts of unstructured data for years. Knowing that Tim Estes, our CEO, might have a different view on this issue, I decided to interview him. Tim has worked within the realms of unstructured data analytics, artificial intelligence, and machine learning for the past decade.

The following is our interview:

Jason Beck – In the article, one researcher suggests that developers shouldn’t build analytics robots, but rather “exoskeletal systems”. Do you agree?

Tim Estes - I think that it’s a matter of degree. The range of judgments that a machine can make as a proxy for the human is constantly and necessarily expanding. Even R2D2…

Matthew Russell’s OSCON 2010 Interview about Unstructured Data

Mac Slocum interviews our VP of Engineering, Matthew Russell, about Unstructured Data at the recent Open Source Conference (OSCON) is Portland, Oregon.

Digital Reasoning’s Matthew Russell featured at OSCON

Next week, July 19-23, hundreds of developers, designers, hackers and geeks will gather in Portland, Oregon, for the 12th Annual Open Source Conference (OSCON). According to the Open Source Initiative (www.opensource.org) “The promise of open source is better quality, higher reliability, more flexibility, lower cost, and an end to predatory vendor lock-in”.

“For those who have not been to OSCON, it’s a great technical conference covering the whole spectrum of open source, including Linux, MySQL, the LAMP stack, Perl, Python, Ruby on Rails, middleware, applications, cloud computing, and more”, said Zack Urlocker from InfoWorld. “OSCON always has great keynotes, tutorials, and evening Birds-of-a-Feather sessions. As with many conferences, a lot of the meat takes place in hallway conversations and impromptu sessions”.

Matthew Russell, our VP of Engineering at Digital Reasoning, will be speaking at the conference again this year. Matt will be sharing his insights about Natural Language Processing, advanced analytics, and entity resolution on a massive scale. (See here for more details: http://www.oscon.com/oscon2010/public/schedule/detail/13988)

I recently sat down with Matthew, and asked him about open source, OSCON and his upcoming participation. The following is part of our discussion:

Jason Beck – So, how long have you been involved with OSCON?

Matthew Russell – I’ve attended and spoken…

Security through Obscurity

“Security through Obscurity” is a term often used to refer to security provided by keeping details of a system secret, or by making a system so obtuse that it is difficult to determine how it works, thus hiding its vulnerabilities. Unfortunately, I believe that there is also an application of this term to the need of identifying and tracking the important information hidden in the mountains of digital data generated each day.

While technology has provided several good paradigms for dealing with structured data (i.e. data that is structured in such a way to be easily decomposed into pre-defined fields), it has not kept pace with unstructured data, such as emails, blogs, web site content, etc. Thus, critical information is often kept “secret” through the obscurity of the sheer volume of data one must process, often manually, to reveal this information.

In response to this challenge, Digital Reasoning Systems, Inc has developed a comprehensive set of analytical tools packaged into product called Synthesys™ that essentially decomposes unstructured text into meaningful information easily understood and manipulated by a user.

This technology is based on the premise that there is order inherent in all languages that can be discovered and mathematically modeled. This has led to…

Eight Things I Learned While Helping Tennessee Flood Victims

On Tuesday, May 4th, forty-eight hours after the worst natural disaster hit our area, our company meeting seemed less important – we paused. Our company’s president, Rob Metcalf, stopped the meeting and redirected our conversation. In that moment we simply could not go about business as usual.

Each of us in the room had been affected directly or indirectly by the floods in Middle Tennessee. We knew we had to do something, we just weren’t sure what. But, rather than form a committee we empowered our employees to do what they thought would make the biggest difference. For some, it meant giving money to various charities like the Red Cross or Hands on Nashville, for others it meant supporting our community by buying “I Love Nashville” flood t-shirts, but for me it meant a trip to the country.

So, on Wednesday, May 12th, my wife, two children, and I travelled to Centerville, Tenn., which is located in Hickman County, one of the counties placed on the federal disaster relief list. My wife and I pulled our son, Carter, out of school for the day. We did so, because we sincerely believe that, in addition to his classroom instruction, we must cultivate civic responsibility…