Metadata Driven Architecture

Software designers and engineers use metadata (AKA data about data) to abstract function from logic.  Metadata creates a logical self-describing framework for allowing the data to drive application features and functionality.  You will commonly find metadata inside of database schemas, XML configuration files, SOAP WSDLS, HTML web pages, rules engines and source code.  There is a wide spectrum of usage spanning from a traditional implementation where a few elements of an application are represented via metadata (i.e. color and size of text), to maximum utilization where the entire application is driven via self-describing data.

In its truest form, metadata driven architecture (MDDA) requires a huge mind-shift.  At its core, MDDA is a deviation from hard-coded logic, to one of dynamic rendering and execution of user interfaces, business rules and workflows. Many systems have elements of MDDA, but never realize their true potential because they lack a strict adherence to the principles of dynamic rendering. As soon as an exception to the MDDA is permitted, traditional development begins to leak into the design. Once this happens, the flexibility of the MDDA becomes diluted, ultimately creating a hybrid of complex and hard to support source code.

Required elements of an MDDA are hierarchical metadata, instance data, a multi-tenant schema and processing engines, which blend the meta and instance data into a logical representation of a function per tenant. Meaning applications can be developed per tenant, using the same processing engines running within the same application container.  There’s some industry debate over where the rendering engines should reside. Client side rendering facilitates a lightweight, client-server architecture as well as fast execution times. The other option is server side rendering and processing.  This creates more network traffic and layer complexities, but ultimately is easier to create ubiquitous reusable rules processing engines. This is a key decision point, deserving of a blog post unto itself.  

The ability to modify the metadata via a user interface (UI) should be one of your end state objectives.  Once this is accomplished, efficiencies in new product development as well as custom modifications per client will begin to become a reality.  I have found that the UI tools are the litmus test for proper metadata design.  If you can’t easily build a tool to manipulate the metadata, then you need to revisit your design.  A really cool byproduct of this strategy is self-generating tools. It is the ultimate recursive paradigm. Imagine building a dynamic rendering engine, driven by metadata, with create, read, update and delete (CRUD) capabilities. Then using that rendering engine and new metadata to build UI tools which allow you to maintain the metadata for the CRUD rendering.  

To take it one step further and reach metadata nirvana, your UI tools need to manifest themselves throughout the application. Meaning if you render a data capture page that facilitates CRUD operations, an administrator has the ability to alter the metadata for that page, via that page. The tools should be presented (with appropriate credentials) to the user, who can graphically alter the meta and instance data definitions, save them to a sandbox, then publish those changes to production.

As an example, the Benefitfocus Marketplace product was built from the ground up using MDDA. The entire application is rendered and controlled via metadata and rendering engines.  There are UI tools throughout the platform that allow admins to alter the user interface, workflows and associated application functionality. We have also developed other mission critical components that are leveraged by clients across our platform via the multi-tenant paradigm inside of the MDDA.   

Additionally, we have developed an evolution strategy for existing core technologies. Our mission is to logically migrate user interfaces and workflow engines to the MDDA framework.  This has become possible because our teams have dedicated themselves to the vision of a pure MDDA strategy, and at the same time are able to pragmatically establish a roadmap of MDDA elements, folding them into the current architecture one step at a time.

I have found that you must first reach a place of extreme position, which crystallizes the vision for the future. You can then look at your current state and begin making logical decisions on how to attain your vision. MDDA - when designed and built from the ground up - is complex, but very doable given a dedicated focus on staying true to the vision. If you are in a non-MDDA world and want to get to an MDDA world, you must first take the extreme position. Focus on how nirvana will look, and get that vision deep into your psyche.