7/12/16:  How are we assuring pattern capture over time even if it isn't seen ahead of time?

Unfortunately, Big Data is a very broad term. At the core of a lot of the techniques is the ability to notice subtle changes in the data sets that may not be overtly noticeable at the output of a system. For example, in a manufacturing machine a fault could be developing in a motor bearing that has no noticeable impact on the yield or quality of the products being manufactured in the machine. However, if this system is monitored using vibration and other sensors the developing bearing fault could first manifest itself as a slight shift in the vibration signature of a rotating shaft or some changes in the current being drawn by the motor. These can be detected with the right measurement systems. The trick is to know how and when symptoms will lead to a catastrophic failure or yield loss. You can't take the machine offline for every subtle variation in sensor data. Removing false alarms while ensuring prediction accuracy is a very difficult challenge.