Chapter 5
Evaluating an Embedded Analytics Solution
Learn how to pick the right technology and the right vendor for your use case.
Choosing the right solution involves a great deal of research. This includes assessing the technology, understanding the expertise of the vendor, and putting together an A-to-Z plan for success.
First, let’s examine the list of criteria that are critical to the assessment. These include the common technical and non-technical requirements.
First, we will detail the common evaluation criteria in each of these categories. Then, we will walk through the entire process for a successful evaluation.
Increase the adoption rate of embedded analytics. This can be accomplished by providing a broad range of users with a tailored experience that matches their needs and skills. Users will typically fall into one or more self-service personas.
Information Consumers
Preference of a defined experience where they can access core business metrics through dashboards and reports that have been prepared for them.
Content Creators
More knowledgeable workers who respond to ad hoc requests for new dashboards and reports.
Analysts
The need for an exploratory environment to discover insights and create new metrics that drive the business forward.
Empower everyone to leverage visualizations. This will help to monitor KPIs and get a complete view of the business.
Data Visualizations
Inclusive of bar charts, gauges, heat maps, spark lines, and geographic maps.
Dashboards
In formats that are both static and interactive, these showcase multiple visualizations in a single view.
Reports
In formats that are both static and interactive, these showcase tabular views of data.
Provide an optimal user experience regardless of where and how users prefer to access information. Evaluate the compatibility of solutions across different devices and formats.
Web Browser
Users should be able to access all content and capabilities on standard web browsers.
Mobile
Users should also be able to easily access and interact with analytics on mobile devices and utilize mobile features such as touch input.
Exports
Content should be available in non-web formats for printing and offline access, such as PDF and Excel spreadsheets.
Create an engaging experience in which users can explore and interact with their data.
Filtering
Users can choose the data that is important to them and get more specific in their analysis.
Drilling
Users can dig deeper and gain greater insights into the underlying data.
Personalization
Users choose the visualizations and reports most important to them. They can also re-arrange content into their preferred view.
Grow user adoption through the usage of analytics into everyday work.
Workflow Actions
Users can act on specific data sets, by initiating a workflow process on select records or making updates without having to leave the application.
Alerts
Users receive automated notifications when certain actions are carried out or thresholds are met.
Scheduling
Content can be scheduled for delivery on a one-time or recurring basis.
Empower users by giving them greater flexibility in their analysis. Give them the ability to create and format the desired content on their own.
Data Query
Users choose the data sources, tables, and columns in which they are interested – without having to write SQL.
Data Analysis and Visualization
This is an experience that is intuitive. Users can see, understand and visualize the data. This is supported through the filtering of data and the creation of new calculations and visualizations.
Dashboard and Report Authoring
Users lay out dashboards and reports. They share what they’ve created with colleagues.
Extend the value of the data in your app. Provide deeper insights into business trends.
Benchmarking
Users can compare their performance against that of the industry. They can then pinpoint areas for improvement.
Advanced Analytics
Provide the unique benefit of advanced (and often proprietary) statistical models in your app.
Use native connectivity optimized for the data source. Ideally, your primary data source should belong in this group.
Databases
Included are SQL Server, Oracle, MySQL, and DB2.
Modern Data Sources
Painlessly connect with modern data such as streaming, search, big data, NoSQL, cloud, document-based sources. Quickly link all your data from Amazon Redshift, MongoDB, Hadoop, Snowflake, Apache Solr, Elasticsearch, Impala, and more.
Cloud Applications
Included are Salesforce, Carbonite, Forcepoint, DigitalOcean, AWS, Dropbox, and Civis Analytics.
OLAP cubes
Used for multi-dimensional analysis
When a vendor-specific connector is not available, generic connectors provide flexibility with data.
ODBC/JDBC
Used for connectivity.
Web Services
Included are REST APIs.
Files
Included are XLS, CSV, and XML.
Enjoy the ultimate flexibility in data sourcing through APIs or plug-ins. These connect to uncommon or proprietary data sources.
Data APIs and Plug-Ins
Coded in your language of choice, these provide customized data access.
A solution that enables you to (1) connect directly to underlying data sources and (2) cache data from transactional systems has two key benefits. It provides real-time reporting and interactive self-service analysis.
Direct Connect
Query directly to the data source for reporting in real-time. Tap into the capabilities of the data source that is underlying.
Data Caching
Unlike “direct connect,” data is extracted from the underlying sources into a high-performance data store. This makes the most out of reporting and analysis from systems that are transactional.
Create a complete, user-friendly view of the data by preparing it for analysis.
Multi-Source Data Blending
Data from multiple sources is compiled and the output is a single view, metric, or visualization.
Data Transformation and Enrichment
Data can be enriched for analysis. Examples include new metrics and calculated values that are frequently used, standardization of dates, aggregations, and manipulation of multi-part text (e.g., addresses).
Metadata
Self-service analysis is made easy with user-friendly naming conventions for tables and columns.
Create an efficient user experience that allows users to immediately act on insights.
Bi-Directional Data Flow
For data updates and workflows, inputs can flow directly back to the source systems.
Incorporate data from external sources into a single consolidated view. Transform your app into a vital hub of information.
External Data
This could be in the form of third-party industry benchmarks, data feeds (such as weather and social media), and customers’ data stores.
By consuming data services from the analytics solution, you can offer flexibility in presentation style.
Data Services
The analytic solution provides both the raw data and functionality. These data services produce outputs to be used by jQuery components, third-party charting, and other application functions.
Making sure that security controls are in place is critical. It should be easy to transfer the security from your application to the analytics content. Scrutinize vendors on the flexibility of their security models. Check out how they launch single sign-on. Ask whether or not data needs to be synchronized or replicated between apps.
Authentication
Single sign-on should leverage the authentication of the parent app. This should be taken care of without having to replicate and synchronize user profiles.
Authorization
Roles and rights established in the parent application are passed to the analytics application. This ensures end users are granted the appropriate levels of access.
Application Security
Fine-grained permissions can be applied to end-user visualizations and functionality. These include charts, reports, and dashboards as well as input controls and user functions.
Data Security
Security can be applied to data sources, tables, columns, and rows. This is crucial for multi-tenant applications.
Accelerate development with a solution that has built-in multi-tenancy support. This allows you to create a report once and deploy for multiple customers.
Multi-Tenancy
A single application has the ability to share data access between multiple customers, whether data is stored in the same database and/or in individual databases per customer. Look for the ability to parameterize and tokenize. Look for those that do not require data replication or advanced data modeling. These support multi-tenancy.
Create an improved UX by embedding analytics as a natural part of your application.
White-Labeling
The look and feel of embedded analytics should match your brand and application. The logo of your analytics provider should not be visible.
Embedding API
Content is usually embedded via a JavaScript API. Parameters can then be passed from the parent application to ensure visualizations are rendered in the correct context.
Application Linking
Users can navigate from analytic content to the parent application and vice versa. A common example is clicking on a part of a chart to go to the specific record in the application.
Create the most efficient user experience. Meaning, one in which users can immediately act on what they see, be it visualization or report.
Workflow Processes
Users can initiate API calls to your application from a report or dashboard. They can perform data operations or process transactions the moment they see the data. For example, a user could select a region of a chart and perform an action on the selected records without having to leave the visualization.
Competing through analytics often means delivering unique functionality. Ensure you’ll be able to meet any future requirement with a solution that can be expanded in scope.
Custom Code
For specific presentation needs, see how custom HTML, CSS, and JavaScript can be incorporated. For specialized functionality requirements, understand how custom-compiled code can be integrated into the solution.
Third-Party Charts
For unique charting requirements, understand how third-party charting libraries and components can be utilized and embedded alongside “out-of-the-box” visualizations.
Empower your development team with the tools to quickly create and iterate on embedded analytics capabilities.
Rapid Development
Assess the tools. Determine how quickly you can create content, fine-tune how the content looks and behaves, and make changes to what you’ve done. Understand how to make both small changes to functionality as well as large-scale ones that affect the entire app.
Out-of-the-Box Functionality
A rich set of capabilities – visualizations, self-service analysis, input controls, and UI themes – will accelerate your product development.
Sample Applications
Access to sandbox applications will accelerate both the learning process and adoption of best practices.
Collaborative Development
Embedded analytics should integrate with your source control systems. It supports version control and collaborative development.
Quickly deploy and scale an implementation that is aligned with your current technology stack. Make sure that you have the flexibility to shift as your technical environment evolves.
Web Architecture
The best solution fits well into your web architecture. It reduces the need to deploy technology that is proprietary. Furthermore, it uses techniques that are known for scaling the implementation.
Deployment Style
The greatest flexibility comes from solutions that can be easily deployed on-premise at customer sites, hosted in your data center, and made available in the cloud through such data platforms as Amazon Web Services and Microsoft Azure.
Software licensing terms should align the vendor with the value that you provide to your customers.
Licensing
Terms of the license can depend on a variety of factors, i.e., number of users/customers, servers, and usage. These may be further impacted by whether you are embedding into a commercial product. Be sure the terms make business sense for the short and long term.
Completing your project on time and in the right way can require resources outside your team. Take comfort from a full range of service options even if you do not employ them.
Pre-Sales Technical Support
Leverage pre-sales resources to fully evaluate solutions. This experience will give you an indication of the vendor’s commitment to you as a customer.
Professional Services
Whether you simply need to augment your staff with a consultant or require a whole team to complete a large scope of work, assess the range of professional services offered. Think about the extent of the partner network.
Training
Virtual and instructor-led training options will bring your development team up to speed quickly. It will help them gain a firm understanding of best practices.
Vendors should supply a process that maps your path to success. They should provide a wide array of resources to address any issues along the way.
Onboarding
Look for a process that quickly engages your team in the solution. This will align resources so you have a clear path. Set milestones for completing each phase.
Account Management
Expect dedicated resources that proactively manage your account. They will update you on trends and can be relied on to handle your questions.
Documentation
The quality of documentation is another sign of a vendor’s commitment to your success. Read carefully.
Support
A combination of live and self-service support options plus backed by professionals should be readily available. They will walk you through any technical issues. Service-level agreements (SLAs) should clearly set expectations for response times.
Community
An active user community can lend peer support. They can share best practices so you can benefit from the experiences of others.
Leverage your vendor’s experiences to make you and others like you successful.
Company Expertise
Inquire about the vendor’s history with embedded analytics. Learn about the resources dedicated to partnering with software providers (OEMs).
Product Roadmap
Inquire about future product releases that will be of benefit to you and your customers. Participate in annual user conferences for up-to-date information and informative content.
References
Ask to speak to existing customers in similar verticals.
To get where you want to go, write it down. Statistically speaking, you increase your likelihood of success simply by putting your goals on paper.
Draw from the strategic benefits we discussed earlier in Chapter 2 (Embedded Analytics: No Longer a Want but a Need).
Identify the steps you’ll take to reach your goals. Ask yourself, “When do I want to…”
Pick the stakeholders who need to be involved. Who is going to care about embedded analytics inside the business (your executive team, product management, lead developers)? And outside (your key customers, customer advisory board)? Build your business case as a team to secure buy-in.
Review your technical and non-technical requirements. Use the previous pages as a guide to rank and weigh the importance of these requirements. Research the competition. Talk to your customers in order to develop a firm understanding of the capabilities you want to add to your application.
First, describe the functional scenarios in which end users will use embedded analytics. Then, map out their goals for each scenario, and turn these into technical requirements. Consider who will use the third-party products inside the company. Understand their skill sets and identify any potential resource gaps as you move into the evaluation phase.
Assign a point person to research each vendor. Assess which vendors’ functionality matches your requirements. Utilize independent industry resources, such as the 2021 Wisdom of Crowds® Business Intelligence Market Study report, to create your initial list. Pay close attention to vendors that specialize in the OEM market for software providers.
Attend product demonstrations by each vendor in order to confirm a basic fit. Discuss your requirements and ask each one to demonstrate how they would deliver your processes and scenarios. Ask tough questions and make sure the vendors show you the functionality they promise. Confirm ballpark pricing to move forward.
Evaluate each vendor’s ability to make you successful. Ask them to demonstrate how their best practices, support, and training will benefit you throughout the implementation.
Avoid a feature bake-off. Meaning, focus on the requirements you identified in step 4 above, and try not to be dazzled by features that don’t deliver on your criteria. Of course, during your search process, you may update your goals as you learn what’s possible. Just remember to stick to the features that will provide value to your customers and that you can really envision yourself embedding into your app.
During your evaluation process, it will be easy to get lost among a dizzying array of charts and graphs. Don’t forget everything we have discussed in this guide. Ultimately, you want to bring the most value to your application, your organization, and your users.
Narrow down your list to the top two or three vendors. Begin a structured evaluation process with each one. This is where you’ll define proof of concept. Establish clear-cut guidelines for what you want to accomplish within a reasonable timeframe of, say, thirty days.
The amount of interaction you have with each vendor is based on your preference. This can be an assisted trial, where support is generally available if you run into issues. Or, a true structured evaluation, where you and the vendor are building a proof-of-concept together.
Always implement this proof-of-concept in a technical environment that is as close to the production environment as possible. That means it should be connected to your data sources, integrated with your security, and be embedded into your app. If you host a SaaS application in the cloud, do not simply assess desktop tools or run analysis off a cleansed spreadsheet. Do what you expect your customers to do.
At the end of the evaluation, share the output back with your stakeholders. Get prompt feedback and validate your direction.
Now it’s time to find out if your vendor can actually make customers like you successful.
Ask your vendors for references. Solicit feedback from others in your personal and social networks. Look for references that are similar (in terms of size, industry, use case, etc.) to your organization.
Find out whether your situation is similar to theirs. Don’t just ask whether they’re happy with the vendor. Really drill into the functionality the vendor has delivered. Ask about the nature of vendor support and training, the duration of implementation, and any roadblocks they’ve encountered. Examine how the vendor handled any problems or issues.
It’s go time! Choose the vendor you feel most confident in as a partner to reach your goals. Of course, you’ll have to compare and negotiate terms and conditions. Look beyond software and focus on the vendor who gives you the highest chance of success.
Make sure your vendor has the resources to help you, even if you don’t need the help today. Later on, you’ll appreciate being able to test ideas and leverage best practices as your needs evolve.
Get training for those who will be using the platform to create analytics. Build your first set of reports. Work with your vendor’s enablement and consulting teams for best practices.
There’s a lot that can be said here, given the endless possibilities that come from using embedded analytics. For the purpose of time and space, here are a few tips for this phase of your process: