OAK RIDGE, Tenn. — Researchers from Google Oak Ridge and some other companies showcased a data processing concept at an international conference in Denmark.
There, they showed off how digital watermarks could be used to improve the way data is read in devices. They said that the new computing technique could be used to help future researchers study real-time data so they can better anticipate consumer needs, estimate supply and demand and deliver more accurate information.
Researchers said that digital watermarks can be used to make sure new tasks are processed immediately after other tasks are completed. It improves the efficiency of how devices process data, allowing them to go through more data in a shorter time.
To prove this, researchers compared watermarks that two data streaming frameworks — Apache Flink and Google Cloud Dataflow. They found that the watermark strategy in one of the systems offered more flexibility to study data, which means this kind of data processing can be used in more kinds of hardware.
Since it can be used in more hardware, researchers said data processing can be used to analyze and understand increasingly complex events by collecting data from multiple sources.
This can be used by industry leaders to help them sort through large databases, finding patterns in consumer behavior and in supply trends so they can make better-informed decisions. The new data processing concept can also be used in industries beyond supply chain management.
Over the past decade, data management has faced new challenges. Real-time computing and social media interactions have made real-time processing more difficult, while autonomous platforms like self-driving cars have required more technological solutions.
“Often, there are too many complex things we want to track,” said Edmon Begoli, who works with ORNL's National Security Science Directorate. “If you want to capture all the manifestations you’re interested in and know when an event begins and ends across all sources, a concept like watermarking is very important.”