The Healthcare Data Warehouse: Lessons from the First 20 Years

The enterprisedata warehouse(EDW) at Intermountain Healthcare went live in 1998, followed by the EDW at Northwestern Medicine in 2006. Twenty years on, in 2018, analytics and technology continue to drive healthcare’s most significant advancements and daily activities, impacting healthcare from executive decision making to the frontlines of care and patient experience. To understand how the EDW has evolved as a pivotal tool and forecast its future role, healthcare IT participants can learn from the first-hand experiences of healthcare EDW early adopters and champions.

戴尔·桑德斯是山间大学和西北大学的首席架构师和战略师。李·皮尔斯于2008年担任山间EDW的领导。Andrew Winter于2009年担任西北EDW的领导，并于2016年将责任移交给Shakeeb Akhter。

This report is based on a webinar in which Sanders, Pierce, and Akhter considered their past healthcare IT decisions and the broader digital trajectory of health. It’s not an exhaustive summary of the discussion but captures highlights of the presenters’ histories in healthcare IT in three categories—people, processes, and technology—with three central themes:

1. What they did right.
2. 他们会做什么不同的事。
3. Thoughts on the future.

People: The Foundation for Successful Healthcare Data Warehouse

成功的医疗保健IT从合适的人员和平衡社会、技术和领域技能的团队文化开始。如今，医疗保健EDW的强大实力向其开发和实施背后的许多人证明了这一点，但作为早期数字医疗领域的领导者，Sanders、Pierce和Akhter也认识到，他们本可以做出不同的早期员工决策。

What They Did Right: People

Three skillsets—social, domain, and technical skills—have formed a successful hiring framework for EDW teams. Social skills help team members collaborate and problem solve through challenges, rather than taking a defensive approach. Additionally, healthcare IT can’t run on technology skills alone; contributors must also have deep healthcare domain understanding.

In addition to hiring around the three skillsets above, successful EDW leaders partnered with and empowered data analysts from the beginning, positioning them as the primary producers of analytics. Once they proved the value of the EDW and had some solid successes, these leaders engaged other executives in data and analytics strategy and execution. They formed appropriate governance committees to get business leadership involved in caring about data and the use of that data and analytics.

What They’d Do Differently: People

一些早期的EDW采用者忽略了数据和遗留源系统的文化问题，尤其是那些较新的医疗保健系统。Compared with other industries,healthcare data异常敏感，并且遗留源系统团队可能会感到受到EDW的威胁。cio并没有试图替换源系统——事实上，他们经常表示，如果没有源系统，edw就不存在——但遗留团队需要一段时间才能理解和信任这一点。

In another early EDW challenge, different types of health systems (e.g., integrated delivery networks [IDN] and academic medical centers) presented unique challenges for early EDW leaders. An academic medical center, for example, wouldn’t be as culturally prepared to take on data warehousing goals as an IDN. Whereas an IDN might have approached clinical variability reduction, best practices, and evidence-based care, its academic counterpart may have a culture of controlled variability. Early EDW leaders had to pivot and focus more on academic organization’s research needs and less on clinical operations and clinic variability reduction.

此外，一些cio过于依赖矩阵技术资源，尤其是数据库管理员和数据库系统管理员，他们擅长事务数据库，而不是分析。分析是一种专长，所以过多依赖矩阵资源的早期采用者缺乏这种专长。有关分析技能的教育项目(例如SQL培训)可以帮助组织建立有效的分析团队——围绕遍历EDW中的大量数据进行教育，并了解用于特定事情的数据结构和表。

招募临床和操作方面的专家来帮助理解数据也会促进早期EDW实践。拥有技术熟练的人员或具有EMR经验的人员提供了价值;但数据仓库和分析部门中真正了解医疗数据的人可以提高更多的价值。例如，一个知道DRG、ICD-9s和-10s代码以及如何使用它们的团队成员;与索赔一起，数据可以加快开发时间，并有助于引入标准化词汇表等控制。

Thoughts on the Future: People

EDW experts predict that data science will increasingly influence EDW hiring in the future. Traditional SQL programming will remain an important skillset, but as data becomes a strategic corporate asset (Figure 1), those programmers need to start building data science and machine learning skills, as well the non-relational technical skills big data requires. Additionally, a new role, the digitician, will keep the digital profile of the patient constantly updated and maintained more effectively than current methods. Today, a health system only sees a patient, on average, three times per year, which isn’t enough to understand the patient digitally. That digitician would round out that patient digital profile with a full picture of patient health (e.g., environment, socioeconomic status, etc.) beyond the traditional encounter and beyond the EHR.

Moving forward, organizations also need to improve data literacy for their leaders. The C-suite needs to know how to ask for analytics help and which questions ask; for example, how do they actually use the insights to improve decision making and change the processes they want to impact?

Processes: From Design and Code Reviews to the Impact of AI

Effective data warehousing processes are rooted in design and code reviews and lightweight data governance. Advanced analytics (e.g., AI) and innovations in treatment and diagnosis will impact these processes, however, changing the nature and priorities of how healthcare manages data.

What They Did Right: Processes

设计和代码审查一直是(并且仍然是)医疗保健分析的关键部分。EDW的先驱们实施了设计和代码审查，以鼓励围绕安全的可靠性，以及分析的准确性。Ina casereported from the UK, a mistake in the mammography screening algorithms in the National Health System resulted in about 450,000 patients failing to get proper mammography screening. An analytics algorithm error caused the mistake, proving the real patient safety issues associated with analytics and value or design and code reviews.

Lightweight data governance (the lower portion of Figure 2), governing to the least extent necessary for the greatest common good, contributed to the EDW’s early success. Both too little and too much data governance had their pitfalls. There’s a reason, for example, that not every case goes to the Supreme Court in the US justice system. By creating what amounts to a Supreme Court of data governance, then trying to implement the principles and the values of data governance at a very distributed level, organizations succeeded with a lightweight approach.

What They’d Do Differently: Processes

EDW pioneers may have benefited from a different approach to prioritization management, including a governance process to formalize prioritization. They would have needed to carefully balance these goals with actual capability, as demand will always outpace EDW capability.

Initially, CIOs also could have better managed expectations around data and analytic quality validation. Revealing data too early to clinicians and researchers might have set unrealistic expectations for early EDW impact.

Thoughts on the Future: Processes

AI and data science will impact major changes in terms process, edging out data governance. Process is going to be more about algorithm and model governance, which will make analytic validation very challenging. There’s a different notion of dev ops when it comes AI, and IT professionals will need to learn what it means to apply dev ops concepts to AI machine learning algorithms.

Changes in diagnosis and treatment will also impact data and analytics processes, as bio-integrated sensors will increasingly enable diagnosis and treatment and put more data in patient hands than in the healthcare system. To keep up, health system data and analytics platforms have to constantly update and upload data to cloud-based AI algorithms. Those algorithms will diagnose the patient’s condition, calculate a composite health-risk score, and recommend options for treatment or maintaining health. The algorithm will also suggest providers based on variables such as quality of care, volume of care, etc. In addition, the algorithm will allow the patient to socially interact with other patients like them, extending the patient’s resources.

Technology: From Late Binding and Beyond

Late binding was a critical innovation in early healthcare data and analytics. Today, modern demands and capabilities require even more agility, as well as advanced security capabilities.

What They Did Right: Technology

Successful early EDW leaders ignored the Enterprise Data Model (EDM) in favor of late binding. In a fluid environment, an EDM is outdated as soon as it’s complete. Also, due to the nature of the EDM process (continuous modeling and mapping), data architects never finish mapping. Every time there’s a change in the environment, they have to go back and change the model, the ETL, and the downstream analytics. An alternative to EDM, late binding doesn’t require expensive ETL tools, as most of the ETL is more object oriented further downstream in smaller grains, not the massive ETL required to EDM.

In addition, effective EDW leaders recognized the value from Microsoft in the early 2000s. They took a risk using Microsoft products; even when their organizations historically used other software and technology companies; that risk has paid off with today’s the more manageable, more automated Microsoft-supported EDWs.

What They’d Do Differently: Technology

Even though late binding was an asset in EDW development, architects may have relied too much on late binding at times, creating challenges when it comes to data modeling. If users only practice binding, they get a proliferation of data objects in the database that are hard to manage. Performance issues (e.g., load time and load management) emerge.

Late binding works well for a while because it’s agile and adaptive. But without an effective way to manage so many data objects, and without reusing some of those objects when necessary, data inconsistency and governance problems emerge. Today it appears that EDW users may have relied too much on late binding.

尽管医疗保健是动态的，但仍然存在一致的数据结构。例如，CMS措施往往具有较低的波动性。通过在企业绑定和后期绑定之间的中间空间绑定到这些数据结构，EDW可以执行更有效的分析。更大的重用和支持还可以提高数据治理效率，结果更加一致。

Another early EDW misstep was too much faith in an enterprise standard business intelligence (BI) tool. Standards provide a common BI tool and provide ease of maintenance, but a common tool also puts constraints on data interaction, which doesn’t allow the agility for effective analytics. Data scientists need the freedom to use the BI tool that works best with their processes and needs.

Thoughts on the Future: Technology

Read-only batch-related warehouses are already outdated. The industry needs more real-time capability—slow and constant trickles of data into the enterprise warehouse platform. Batch loads cause huge performance spikes on the source system, as well as the data warehouse, and lead to slow decision making. Data uploads every day, or every week, don’t support timely decisions.

Future warehouse platforms must have cloud-based hybrid transaction and analytic architectures. For this reason, today’s Health Catalyst® Data Operating System (DOS™️) is cloud based (namely, Azure). The cloud offers unmatched agility and security.

Figure 3 shows a typical modern healthcare data warehouse architecture. Data sources are on the left, with different file structures feeding into the platform. Data integration breaks up into three parts, and AI tools are a natural part of the pipeline. The lower levels contain compute and AI clustering, the transaction data storage. And, over on the far right, arrows now go back into the platform, whereas in a traditional data warehouse it all goes from left to right. This modern architecture has the ability to write applications back into this environment. Future applications will support work flow, providing a hybrid combination of analytics and work flow in the same user experience.

Twenty Years Later, Healthcare Data Warehouse Architecture Is Still Evolving

Sanders opened the presentation admitting that trial and error defined his analytics and data warehousing journey. “Generally speaking,” he said, “I learned what was right by first doing what was wrong, in life in general, but certainly in analytics and data warehousing.” Even though Sanders, Pierce, and Akhter achieved many early wins that shaped the course of the EDW and healthcare analytics and data warehousing today, their initial missteps are equally influential in the present and future of the industry. With analytics skills and technology ever advancing and innovations in diagnosis and treatment transforming care delivery, analytics and data warehousing leaders who maintain a similar spirit of agility and humility will have the biggest impact on outcomes improvement.

Additional Reading

你想了解更多关于这个话题吗?Here are some articles we suggest:

1. Cloud-Based Open-Platform Data Solutions: The Best Way to Meet Today’s Growing Health Data Demands
2. Healthcare Analytics Platform: DOS Delivers the 7 Essential Components
3. The Homegrown Versus Commercial Digital Health Platform: Scalability and Other Reasons to Go with a Commercial Solution