O

Services

Case studies

Contact

25.10.20

Data infrastructure is maturing

featured image thumbnail for post Data infrastructure is maturing

As data becomes the key to businesses success, the infrastructure that supports it is maturing.

Nearly all companies have some sort of data infrastructure. This can start with a few excel spreadsheets held on laptops, and can grow to huge data centres managed my internet giants like Google. This infrastructure can serve two main purposes:

"To help business leaders make better decisions through the use of data (analytic use cases) and to build data intelligence into customer-facing applications, including via machine learning (operational use cases)."

Broadly, the analytics use case tends to be centred around a data warehouse, and the operational approach tends to focus on a data lake. This article gives three useful blueprints for data architectures, depending on your business requirements.:

  • Modern BI (cloud based data warehouse + visualisation tools - suitable for most companies)
  • Big Data (big enterprises or tech firms with complex needs. )
  • AI/ML (the least mature and most difficult to get right, many companies build their own. Lots of open source projects. Many don't need this.)

BEWARE: the article linked is written by A16Z, who have a significant investment in data infrastructure company Databricks, that happens to be at the centre of most of these diagrams. Despite this, it's a useful, if slightly biased, introduction.

←Previous: Keep your brain on its toes

Next: A new way to understand reinforcement learning→


Keep up with the latest developments in data science. One email per month.

ortom logoortom logoortom logoortom logo

©2025

LINKEDIN

CLUTCH.CO

TERMS & PRIVACY