Is your organization 'Data Driven'?

Does your organization collect data that looks like this?

[Removed and replace barring in the machine took a while I followed the spec but found a loys of issues needed to order part 12345 in place of 12456 due to quicker instalation]

Does this scenario sound familiar?

You are a manufacturer of widgets with 100 machines in plants across the country. Your organization has many unscheduled downtimes for no obvious reason, and these events are costing your organization time, money, and your good reputation with your customers.  Your gut is telling you there is a problem with your machines.  In order to validate your hunch, you pull 5 years of sensor data (structured data) that monitors your machines, and you correlate your technician’s pm, or periodic maintenance data (unstructured data).  You give the project to your business analyst who is trained in pattern matching algorithms, and by using some simple Excel tools, such as regression analysis, your business analyst determines that the downtime is due to the technician’s periodic maintenance (pm) schedule.   But you need more information. 

It can sometimes take a significant investment to become data driven. Building a repository of over 5 years of data entails significant infrastructure and the ability to orchestrate a well-coordinated capture of data at differing intervals. Determining when to increase/decrease polling intervals of sensors or logs can impact the size of the IT infrastructure required to support the massive amounts of data. The downside of less frequent polling intervals is the risk of a reduction in data outcomes produced. Some other factors include the ease of access to structured (databases) and non-structured data such as (logs, technician work orders).  Once you have identified the data types and sources needed, you might select one or two projects that will add value to your organization.  From there you can measure the outcome of the project and validate the approaches taken.  

In the above scenario we show examples of structured data from a collection of data, called a Data Historian (a system that records and retrieves production and process data by time), and data from technician work order system in an unstructured format. Structured data typically is stored in a relational data form which is very easy to read as input into a computer.  On the other hand, unstructured data is non-relational and therefore very difficult to read as input.   This is due to the behavior of a technician during the act of entering a maintenance note which is typically written for their own information, in their own words or abbreviations. Unstructured data, in this case, is what the technician types into a free form field.  

At this point, a data scientist could further investigate and discover that there is a pattern which correlates downtimes and the pm, and determine exactly how often this is happening as well as its impact on the business.  Prior to this analysis, the pattern match was too infrequent to notice or simply determine by a hunch.   The consequences of such obscure patterns are very costly.  In this scenario of a manufacturer trying to understand the cause of pesky downtimes, there is a real world example of how businesses are relying on hunches to solve problems, rather than adopting a culture of data.

What lessons did this organization learn from its data?

  • What variables are important to the discovery of the machines’ downtimes.
  • The more data you have, the clearer the outcomes.
  • Which tools are required to view all of the structured and unstructured data.
  • The accessibility to sensor data at a proper polling frequency with the ability to increase polling of sensor based on an established set-point.
  • Your gut was wrong, therefore you need a data driven analysis.
  • You need a resource that is trained in the use of methodical tools to properly interpret the data.

The following concepts are some of considerations your organization may need to implement prior to becoming a data driven culture:

  • Your organization needs to store 5 - 7 or more years of sensor data which may not always be reliable.
  • Do you have the right amount of sensors monitoring your machines and are they polling at proper intervals?
  • Does your manufacturer provide data baselines and patterns to match?
  • Do you have the ability to correlate the structured and unstructured data to use as a common data set?
  • What are the required methods of capturing unstructured/structured data and archiving the data?
  • Determining what variables are within the common data set will help determine a pattern match.
  • How can you eliminate ‘noise’ within the dataset?
  • Do you know the required algorithms which can be applied given the data you have?
  • Determine if you need a resource or consultant that can help provide tools and insights needed to make recommendations derived from the data outcomes.

So where should you start?  Start with a systems approach to your business.  Understand your workflows, inputs and outputs.  Analyze your workflow and data to understand which algorithms can be applied and what kind of value they can produce. Start off with the most important opportunity, and build up from there. This effort will take some time to establish, and a significant investment within your organization will be required.  Your organization may need to make radical changes to become truly data driven.

Author: Sam Marrazzo
With over 25 years of IT experience with a focused in Manufacturing and Supply Chain.

IT WNY GOLD Sponsors

  • is your direct link to the MLS. WNY’s most reliable and accurate Real Estate source! The only place for every open house, every day of the week! 

  • 360PSG

    Making Web Development More Accessible To You.

    Our mission is to provide web solutions in such a way that small and mid-size companies have an affordable option, without sacrificing the quality and expertise that is usually associated with large, diverse and expensive agencies.

  • At AT&T, we’re bringing it all together. We’re helping people connect with advanced mobile services, next-generation TV, high-speed Internet services and smart solutions for businesses.

    Fast, highly secure and mobile connectivity – to everything on the Internet, everywhere, at every moment and on every device – is what drives us at AT&T.


  • Blue Cross Blue Shield WNY

    We strive to provide better healthcare to our members, offer greater value to our customers, and improve the health of our communities.

  • We will maintain an unfailing commitment to understand our clients' needs and to provide comprehensive, practical, high-quality, and responsive solutions. We will maintain the highest professional and ethical standards, and continue the Firm's historical commitment to, and leadership in, community activities, pro-bono work, and service to the Bar.

  • Buffalo Business First

    View Daily Local Business News, Resources & more in Buffalo, New York.


    FCS specializes in digital media design and development, taking a project from initial conceptualization through scriptwriting, into digital video/audio recording, and culminating in non-linear, uncompressed editing. With advanced capabilities in 2D & 3D animation, graphic design, and illustration, projects include broadcast television shows, training programs, marketing presentations, digital signage, and fundraising support. FCS can master all professional formats including 4K, web-based applications, and others. 


    DataSure24 – our services accelerate and simplify continuous monitoring threat detection, incident response and compliance management for IT teams with limited resources. Protecting your data can be one of the biggest challenges for your business. Our clients can rest easy knowing that their data is protected 24/7 by our team of specialist SOC Analysts.

  • Data Breach Defense

    Cyber Security, Breach Readiness, Risk Mitigation, Data Breach Compliance and Notification Services.

  • We support our clients by providing solutions that leverage industry leading technology and services.  We have been reducing risk and expense from IT based projects and day to day operations since 1965.  Headquartered in Tonawanda, NY, Ronco has grown to over 25 locations in the Eastern United States.  Service is our primary focus and our commitment is what brings us together.  We have built the foundation of our organization on trusted partnerships throughout our footprint.

  • TIBCO fuels digital business by enabling better decisions and faster, smarter actions through the TIBCO Connected Intelligence Cloud. From APIs and systems to devices and people, we interconnect everything, capture data in real time wherever it is, and augment the intelligence of your business through analytical insights. Learn how TIBCO makes digital smarter at

  • Rich Products

  • Twisted Rope is a full-service digital agency with offices in Buffalo, Los Angeles and Netherlands. Twisted Rope works collaboratively with companies to drive their marketing, sales and business development strategies. From branding to creative content, software, and app development, Twisted Rope combines strategy, design, and development to create engaging experiences.

  • Lighthouse Technology Services | IT Staffing, IT Services, IT Jobs – Introducing Companies to the Right Technology Professionals and the Right Technology Solutions, at the Right Time.

  • Beckage PLLC is a law firm focused on technology, data security and privacy law, with offices in Buffalo and NYC.  Beckage attorneys are CIPP/E and CIPP/US and assist their global clients with all matters related to technology and privacy from regulatory compliance and contracting, data breach response, and representing clients in national putative class actions and privacy litigations.  


    At M&T Tech, we’re a team of makers, doers, and builders working to create the most advanced technology solutions in banking.  We’re not your stereotypical suit and tie bankers: we’re an innovative team of leading tech experts, pushing boundaries, and taking risks. Join us and be part of something new as we build tomorrow’s bank today.

  •  The PMI Buffalo Chapter was founded in 1995 to support the visions of PMI (Project Management Institute) in the Western New York area.

  • For over two decades, iV4 has been aligning our customers with tailored technology solutions that enable them to modernize, protect and grow their business. We are proud to be the leading Microsoft Partner in Upstate, NY and honored to have been named a Top 200 Microsoft Solutions Provider, Microsoft Partner of the Year and Cloud Partner of the Year. We do technology best, so you can do business better.