Avoiding Storms in Your Move to the Cloud
By Ravi Chandran, Norman C. Nicholl,
and Tracy Ring
Stacey Bostian is the BI director at Superior Lighting, a leading manufacturer and seller of lighting products for home, office, and outdoors. Stacey has been with Superior Lighting for 10 years and was responsible for creating the 10-terabyte data warehouse, and selecting and implementing a leading BI tool to generate reports, dashboards, and mobile applications.
Stacey’s team has dabbled with BI in the cloud, using her BI vendor’s cloud-based platform to develop several applications. Once successfully developed, the applications were moved on premises, where everything else currently resides. Based on these successful SaaS experiences and the belief that the cloud is the future, Stacey is interested in exploring what, how, and when she should start moving things to the cloud. There are currently discussions within Superior Lighting to move other data and applications to the cloud, including its ERP system.
She does have questions that perhaps you can help answer...
RAVI CHANDRAN
Stacey is clearly at a stage where the cloud seems an attractive option, so she is cautiously considering next steps—a stage common to many these days.
From the perspective of one who offers an analytics platform on multiple clouds, the first issue I recommend she guard against is being locked in to a single cloud provider. The software you use for your BI and analytics stack should give you maximum flexibility, which includes on-premises deployment as well as multiple public clouds.
As Stacey is probably well aware, there are strong competitive pressures between the major cloud providers (AWS has had more than 50 price reductions to date), and we can expect further differentiated offerings in terms of both features and price. Stacey should position herself to fully leverage these new offerings by planning ahead for migration from one cloud to another.
It appears Stacey has had a positive experience with the cloud so far. She has found that the cloud enables easy and rapid application development. These features—agility, low-risk, low-cost, try before you buy, and pay-for-use—are the driving factors behind cloud adoption. They are also probably the motivation for other groups within Superior Lighting (such as ERP) to move towards the cloud.
It is much easier to plan for operational systems—for example, to build out capacity in your data center for the next three years. This would be nearly impossible for analytics systems.
However, unlike operational groups, an analytics group has much more demanding requirements. Growth in data volume and data sources for analytics (social media, location, online ads) dwarfs that for operations (transactional data). Therefore, it is much easier to plan for operational systems—for example, to build out capacity in your data center for the next three years. This would be nearly impossible for on-premises analytics systems; they need the elasticity of the cloud.
On the flip side, on-premises systems offer a familiar landscape of tools (no learning curve) and may provide reduced operational costs in the long term, albeit at the cost of up-front capital expenditures.
Increasingly, though, the same onpremises tools are also being offered in the cloud, either by the same tool vendors or by the cloud providers themselves. This is certainly the case for identity and access authentication tools, such as Microsoft’s Azure Active Directory and AWS’ Identity and Access Management (IAM).
Similarly, there are numerous solutions for data security available. Cloud providers support encryption of data at rest and associated services for key management (AWS Key Management Service, Azure Key Vault). They also support the concept of isolated networks (a “virtual private cloud”) and isolated machines not shareable by other cloud users. Additionally, there are numerous commercial and open source solutions available for ensuring secure network access to the cloud from a desktop. In short, access control and data security should not be stumbling blocks to Superior Lighting’s cloud migration plans.
Since Stacey’s analytics group has to deal with large and growing volumes of data, her top three priorities should be storage, storage, and storage! To most people, cloud evokes the image of on-demand computing via “virtual machines.” This is partially correct, but it ignores the significant storage options that the cloud also provides.
Clouds offer several tiers of storage, each with different features and price points. Broadly, they can be classified in order of increasing cost and flexibility of access as “Archive, Object, Block” (these options are explained in more detail in the 2015 article, “DW-On-Demand: The Data Warehouse Redefined in the Cloud,” Business Intelligence Journal, Volume 20, No 1, pp. 8–13). From the perspective of a BI user, you should view analytics platforms as systems with decoupled compute
and storage. Storage in the cloud offers many more features than storage on premises at dramatically lower costs.
For instance, replication across geographically separated zones is available for relatively low marginal cost. This means your data is protected and available for disaster recovery. Decoupling compute from storage enables recovery and independent elastic scaling of both storage and compute.
Another significant benefit derives from the relative cost difference between cloud storage and cloud compute. Storage cost is very low ($25–$50 per TB per month) in comparison to compute cost ($1–$5 per machine per hour).
With decoupled storage and compute, data can be safely maintained even when compute resources are shutdown. This avoids the per-machine-per-hour costs during idle times (nights, weekends, and holidays) and can result in cost savings of more than 60 percent.
The elasticity and ease of use that the cloud provides could also cause some headaches, though. Without proper governance in place, there is a danger that users in Stacey’s group could rapidly propagate cloud deployments and run up significant usage bills without her knowing—until the invoice comes, of course! Stacey should implement internal processes to control and monitor cloud usage, ideally before her first real cloud project is undertaken.
NORMAN C. NICHOLL
To help Stacey take her company’s business forward into the cloud and leverage her team’s experience delivering on-premises BI solutions, I’m going to walk through some key cloud discussion topics.
To prepare for the discussion with senior management, Stacey needs to understand the overall corporate cloud strategy and which cloud vendors Superior Lighting may want to use for hosting.
Getting educated and connecting with someone who has experience working with the identified vendors will go a long way toward understanding (at a high level) how the cloud will fit with the current BI implementation and will help identify any capability gaps.
With that knowledge about the overall direction and fit of the current solution with the cloud, Stacey can have a more meaningful dialogue with senior management around cloud corporate strategy—including timing and priorities for systems and applications migrating to the cloud.
Getting educated will go a long way toward understanding how the cloud will fit with the current BI implementation.
This is also an opportunity to educate management on how moving the BI infrastructure to the cloud can improve the company. This includes explaining new capabilities the cloud brings for operational integration, mobile applications, increased agility, and so on. In this way, she can paint a vision for next generation BI capabilities at Superior Lighting.
Her willingness to move quickly with some portion of cloud deployment—even if it’s only development and testing—without putting the business at risk will help accelerate overall corporate adoption.
The question of on-premises versus cloud remains an interesting one, though. Continuing operations on premises allows you to maintain the status quo with reduced short-term risk while maintaining existing development and deployment life cycle and budgeting policies. This ensures resources are focused on immediate business and technical needs.
However, here’s why cloud still matters:
- On-premises operations require continued capital expenditure budgeting and paying for peak demand; in contrast, the cloud only requires you to pay for what you use, which usually comes from the operational expenditures budget.
- Enabling new capabilities on premises requires hardware and software investment, which can take time
to acquire and implement; in the cloud, you can try new capabilities cheaply by instantly spinning infrastructure up and down to prove business value. - Capacity caps due to limitations with on-premises infrastructure (such as power, cooling, network or hardware capacity, and licensing) can impede business growth; the cloud brings agility to better support the ebbs and flows of a business.
So why does moving to the cloud matter? Data in motion takes time, introduces analytics latency, and costs money. Therefore, having the BI infrastructure and processes close to the data provides many advantages.
Once in the cloud, the business can take advantage of more seamless integration and enhanced capabilities. In addition, sharing governance, standards, and procedures with other business units going to the cloud will help reduce the learning curve. The analytics team must be part of this process or they risk quickly becoming a “legacy” team.
Move the current warehouse technology stack if—and only if—it is optimized for the cloud and can take advantage of the real benefits of moving.
What should Stacey do with her on-premises warehouse? It should eventually be migrated to the cloud—assuming the underlying technology supports it and there are no other mitigating factors. Although warehouse options in the cloud are abundant, many are still maturing.
I would suggest moving the current warehouse technology stack if—and only if—it is optimized for the cloud and can take advantage of the real benefits of moving to the cloud (e.g., flexibility, agility, scalability, cost). Alternately, she can look at new platforming options during the migration, if time and funding can support it.
Stacey should consider starting with a hybrid approach. With the warehouse on premises and BI in the cloud, she’ll have time to analyze the broader impact on capabilities, governance policies, sourcing, ETL, and downstream users. Choosing a hybrid implementation to start could be a more agile, risk-averse approach.
Stacey’s inevitable security discussions must be based on reality, not “blue sky” wishes, so engaging the corporate information security or governance team early is critical to keeping the conversation fact based and accurate. The information security team can:
- Review and validate current on-premises data and security practices for applications, infrastructure, network traffic, encryption, and so on
- Capture any new near-term security requirements, including simplifying existing processes
- Map those requirements to cloud provider capabilities and support to ensure the security provided by the cloud vendor will be equivalent or better than the current on-premises solution
Stacey’s inevitable security discussions must be based on reality, not “blue sky” wishes, so engaging the corporate information security or governance team early is critical to keeping the conversation fact based and accurate. The information security team can:
What conversations should Stacey have with her current BI vendors? To begin with pricing is different in the cloud and understanding licensing options ahead of time will help. Questions such as “Can I bring my own license?” and “Does the pricing model support scaling up or down on an hourly basis?” can make a big difference to the financial model.
Improvements that a vendor has made to their products or services to specifically take advantage of cloud capabilities (micro services, dynamic scaling, provisioning, automatic routing, global availability, and so on) should be another important consideration.
Transitioning to the cloud may have pitfalls, of course. It is critical for Stacey to understand the technologies and capabilities of the cloud, even if Superior Lighting’s BI doesn’t ultimately move to the cloud right away, because all BI will likely need to interoperate with the cloud at some point. She would also benefit from getting on board early with the company’s cloud strategy so she can make a meaningful contribution and ensure her point of view and requirements are being met.
Understanding how pricing in the cloud works for her requirements is another critical component. In some cases, pricing for compute power, disk space, and software can be deceivingly low—until you calculate those costs over time for a 24-hour-a-day, 7-day-a-week fully functional production environment. Stacey may need to look at alternative cloud solutions to help keep the cost down.
Finally, don’t get too enamored with the new technologies. There are many and they’re maturing quickly, so be sure they give you the agility, features, and scalability you need when you need it, at the right price.
Don’t get too enamored with the new technologies. There are many and they’re maturing quickly, so be sure they give you what you need.
TRACY RING
When it comes to management discussions, Stacey should first and foremost be working with her CIO or chief technology leader. Many CIOs and CTOs have adopted a cloud-first strategy and aligning with their overall vision can only strengthen her case.
She can take this opportunity to discuss her experiences and success using the cloud. She can also explain that, while she was waiting for the cloud to mature, she was testing its capabilities and is quite pleased with the results. Based on these successful SaaS experiences and the belief that the cloud is the future, Stacey can build a strong case for why a move to the cloud should be a strategic consideration.
When presenting this to the larger team, she may want to help the executive team by showing how often the cloud is used in their everyday lives. They are most likely already using cloud services to solve routine problems without even knowing it.
Sharing personal use cases where her peers are entrusting the cloud—such as family photos, personal email accounts, group calendars, and other cloud solutions—will help Stacey build connections with the team and encourage a move to the cloud.
There are several benefits of moving to the cloud:
- Lower total cost of ownership
- New capabilities and better business value
- Alignment with future state architectures and modernization readiness
- The ability to attract top talent with an eye for innovation, differentiating IT as a strategic capability
- Focusing company resources on development and high-value activities versus hardware and maintenance
However, a move to the cloud undeniably changes the face of corporate IT. In some cases, it may prove challenging to individuals who feel threatened by the shifting workplace paradigm. Even so, as with previous technological shifts, Stacey should see this as a positive change and an opportunity to communicate to the rest of Superior Lighting that the move to cloud solutions actually opens up new opportunities for the IT staff.
Stacey should also consider the consequences of being fixed to one provider. The perception of being “locked in” is a huge barrier. To mitigate this risk, Stacey should conduct sufficient due diligence and work together with her procurement organization or a consulting firm to help her through some of these negotiations.
Knowing that other data and applications might be moving to the cloud, Stacey will continue to see vendors using a cloud-first strategy. In most cases, once a CIO understands the value proposition of the cloud and sees the potential decrease
in overall risk, he or she will open up more opportunities for IT to focus on development and other tasks instead of traditional ROI.
Concentrating her strategy on where IT vendors are spending more will ensure that Stacey’s architecture stands the test of time. She shouldn’t shy away from asking about a vendor’s R&D budget—it’s a leading indicator of where an organization is investing and, of course, one of the best ways to understand the direction of future architecture developments.
What should she do with her onpremises warehouse? As Stacey will soon realize, most companies are not quite ready to bet everything on the cloud. They want certain features of the cloud while maintaining the comfort and control of having their data on premises. This concept of a hybrid cloud is gaining more traction among IT professionals who want the benefits of the cloud without having to give up all their locally controlled data.
As Stacey will soon realize, most companies are not quite ready to bet everything on the cloud.
One way she may be able to gently encourage change is to include all the enhancements to the data, dashboards, and visualizations in the new platform, while keeping the old platform intact for an 18- to 24-month transition period. The
potential benefit for moving to the cloud would then be to take advantage of new metrics, data sources, and tools that can better enable the business.
New platforms coupled with change management and strong communication may ease the pain of moving to the cloud. Over time, the abilities of the cloud will likely grow and evolve as new vendors establish themselves in the cloud-based infrastructure.
Security is the most talked about concern; in fact, it is currently one of the highest barriers to cloud adoption. Therefore, Stacey must be prepared to respond to any security concerns that her company may have regarding the move. Nonetheless, Stacey’s gut feeling on security is typically in line with what we see from many clients.
In addition to understanding security concerns about the cloud, Stacey should also begin having conversations with the executive who is managing security of their environment (typically the CISO or controller). This will help her develop a better understanding of the DLP tools and other data security intelligence tools that are already in place internally.
Stacey may also be able to leverage what they are already using in building the new infrastructure. An important question Stacey should ask herself in this process is how she can partner with the CISO to drive a cloud-first strategy. This may be an opportunity for her to work with her CISO to best categorize and monitor sensitive data and understand data proliferation. It may also be an opportunity to increase their capabilities around security in the cloud rather than constraining their security capabilities on premise.
Stacey should understand their product road map and their approach to data modernization. As her architecture evolves in the cloud, there may be an opportunity for additional players. Thus it is very important that Stacey look at the
entire platform and not just one or two aspects of it.
Stacey should develop a clear view of new and existing vendors, and how they may come into play as new technology emerges. This is the perfect time for Stacey to revisit her BI strategy, which takes into consideration anticipated technology changes over the next 18 to 24 nmonths, and will be the best way to chart a cloud future.
Although we have discussed the many great opportunities evolving around the cloud, Stacey should not have reservations if certain pieces of her architecture aren’t good candidates for the cloud because of geography or the need for a subsecond response. There are still applications that may need to sit on premises and that is okay. Even in a cloud-first strategy, on-premises applications can still be a part of the architecture.
Stacey should also prepare herself for the likelihood that many people will still want to stay on premises. Most importantly, Stacey should know that the focus on security cannot be underestimated. As mentioned earlier, it is the biggest limiting factor we see in adopting the cloud.
By nature, the cloud is a shared resource. It is a model for enabling convenient, on-demand network access to a shared pool of configurable computer resources that can be rapidly provisioned with minimal management, effort, or service
provider interaction. As Stacey is negotiating performance, she needs to remember that even though her data is secured in the cloud, it is still a shared resource. Stacey should work with a procurement or consulting firm to ensure that all
performance statistics are clearly stated and that she uses proven, qualified cloud providers with reliable service and support.