Process your data stream as you like: Invoke filter operations, enrich streams with data from external sources, aggregate data over sliding time windows or modify existing fields in a stream.
Support for complex pattern detection allows to detect situations you would like to avoid. You can also integrate custom algorithms for more advanced real-time analysis.
Our drag-and-drop interface allows the definition of pipelines in a web-based modeling tool. Our system automatically translates pipelines to an execution model so that you don't need to handle the complex details of distributed applications.
Our technology understands not only syntax, but also the meaning of your stream. This enables us to recommend you processing elements that suit to your sensor input.
Your requirements have changed? Our system can be extended at runtime. StreamPipes asssists you with a web-based model editor and a code generator. Developers only have to add the actual business logic.
As our runtime is built upon mature, open-source-based distributed stream processing technology, you can be sure your processing capabilites are able to grow with your sensor input.
StreamPipes allows to immediately detect incidents you'd like to avoid. We support algorithms ranging from simple threshold-based tracking of sensor measurements over trend analysis over time periods up to the integration of custom-tailored predictive maintenance algorithms.
StreamPipes helps to create a clean data lake based on sensor measurements from machines and other assets. Various data harmonization algorithms (e.g., filters, aggregations and unit converters) allow to easily clean and enrich data in a continuous fashion.
See what's happening right now: Use StreamPipes as your real-time window into your current production performance. A live dashboard and a wide range of available notification channels allow you to monitor KPI's in a flexible and customizable manner.
Easy to use. Web-based user interface to create stream processing pipelines in a graphical editor.
Powerful. Select data streams, add transformation modules and choose a data sink. StreamPipes cares about execution details of your pipeline.
Intelligent. Our built-in semantics recommend you elements you can connect and disallow you to connect elements that wo not work.
Intuitive. No deeper knowledge of schemas, protocols or data formats required. From now on, a data stream is just a circle.
Dynamic. Once your sensor data changes or you need to add new streams, just reconfigure the stream in our web interface.
Versatile. We have successfully integrated data streams ranging from MES systems over mobile phone data to twitter streams.
Data Processing and Transformation
Filter, Aggregate, Enrich. You need less data, more data or other data? Stateless processing components transform your input streams.
Advanced Analytics. Advanced methods (e.g., ML-algorithms) can be used as pipeline elements.
Pattern Detection. We have developed modules supporting pattern detection, e.g., detection of sequences over time.
Visualize. Create real-time visualizations and observe your KPIs and production systems from web-based cockpits.
Notify. Detect problems or other situations of interest and notify the people in your organization who need to be informed.
Store. Harmonize your data stream and store it in third-party systems, e.g. for offline analysis. We have adapters for Elasticsearch, HDFS, Cassandra and NoSQL databases.
Here are the technical details
RDF-based data model to describe semantics of streams, processing logic and sinks
Semantics-based matching based on schema, protocols, formats and data quality aspects
Arbitrary output transformations: Extract, rename, replace, append, or customize the output of data processors.
Web-based assisted definition of new sources, data processors and sinks. Extend the system at runtime. Code generation for supported runtime implementations.
Transport protocols: Support for multiple message protocols and brokers, e.g. Kafka, REST, MQTT, JMS, AMQP or STOMP.
Data Formats: Support for multiple message formats, e.g., JSON, XML, Thrift
Runtime-independent integration of heterogeneous stream processing systems: We have already integrated Apache Storm, Apache Flink, Esper and standalone algorithms.
Intelligent Monitoring: Detection of sensor failures and semi-automatic replacement with backup sensors