Menu
Twitter to open source streaming data analyzer

Twitter to open source streaming data analyzer

Twitter will release complex event processing software it acquired in last month's purchase of BackType

Expanding the field of complex event process software with another offering, Twitter will release as open source its software for analyzing live large-scale data streams, called Storm.

Twitter acquired the software when it purchased BackType in July. BackType offered a service that analyzed the impact of an organization's Twitter feed, summarizing how often Twitter messages were repeated by others.

Although the software has been compared to Hadoop, Storm is best suited for analyzing live data streams, such as millions of Twitter feeds.

This approach could provide a speedier and more practical alternative to the traditional approach of real time analysis, which can involve storing the data first in a database, data store or data warehouse. Its use is not limited to Twitter, however. Storm could be used to study other forms of unstructured, frequently updated data.

"The beauty of Storm is that it's able to solve such a wide variety of use cases with just a simple set of primitives," said Nathan Marz, in a blog posting announcing the pending release.

The user creates a query, or search term, that will continue to run against an ever-updating stream of data until the query is terminated. Because Storm can be distributed across multiple servers, it is capable of analyzing large amounts of data.

This sort of analysis is sometimes called complex event processing (CEP). Oracle, StreamBase, SAP and other companies also offer CEP software. Unlike most of these products, however, Storm does not have a built-in storage layer, relying instead of external data stores, Marz pointed out.

Twitter will launch the software next month at the Strange Loop conference in St. Louis.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is Joab_Jackson@idg.com

Follow Us

Join the ARN newsletter!

Error: Please check your email address.

Tags open sourceapplicationsdata miningsoftwaretwitterBackTypedata warehousing

Upcoming

Slideshows

IN PICTURES: Nutanix's .NEXT channel event in Sydney (+20 photos)

IN PICTURES: Nutanix's .NEXT channel event in Sydney (+20 photos)

Nutanix recently held its customer and channel event, .NEXT, in Sydney. The event, held at the Sheraton on the Park saw attendance from more than 150 channel and technology partners and customers. It was the first in a series of events Nutanix is holding in A/NZ in August and September, the objective of which is to brief partners and customers on “what’s next” in the design and management of datacentre technology.

IN PICTURES: Nutanix's .NEXT channel event in Sydney (+20 photos)
IN PICTURES: EDGE 2015 sponsor debrief (+23 photos)

IN PICTURES: EDGE 2015 sponsor debrief (+23 photos)

Some of the sponsors of ARN's inaugural EDGE 2015 event got together at the ARN office for a debrieef of the event. Over some drinks and cheese, these attendees got an update on some key statistics that arose from the EDGE event and discussed potential topics and improvements that can be made at next year's event.

IN PICTURES: EDGE 2015 sponsor debrief (+23 photos)
IN PICTURES: ARN Distributor Roundtable, Sydney, 26.08.15 (+26 photos)

IN PICTURES: ARN Distributor Roundtable, Sydney, 26.08.15 (+26 photos)

ARN hosted a distributor roundtable at Cafe Del Mar in Sydney, at which attendees and their partners discussed the changing role of the traditional IT distributor. They spoke about the challenges of digital disruption, the blurring lines of the channel in the age of digital transformation, and examined the ever-evolving business models. This roundtable was sponsored by Distribution Central, Exclusive Networks, Rhipe, and Hemisphere Technologies. Photos by ARN Editorial Director, Mike Gee.

IN PICTURES: ARN Distributor Roundtable, Sydney, 26.08.15 (+26 photos)

iasset.com is a channel management ecosystem that automates all major aspects of the entire sales, marketing and service process, including data tracking, integrated learning, knowledge management and product lifecycle management.

Show Comments