What Are the Killer Apps for SAP HANA and Other In-Memory Computing Systems?

In a previous blog, I argued that SAP HANA (and in-memory computing) had the potential to bring a number of benefits to enterprises in the short term, including:

• elimination of lag time between data capture in the operational system and its availability in analytical systems,
• greatly increased query performance, and
• simplification of the IT landscape.

A second blog discussed scenarios in which HANA could be transformative to customers today. In summary, customers running SAP BW may find substantial benefits to moving to SAP HANA in the short term - read the blog for more details. It's my opinion that SAP BW is the "killer app" for HANA. However, this is only a part of the answer, since BW is a platform on which customers run many different apps.

"Timeful" software

Why is HANA so interesting? In a sense, what the HANA team did is to look at all the assumptions underlying applications today. Given the enormous changes in the price of high-speed memory, it is now possible and economical to handle essentially all of our typical transactional applications - and a very large fraction of our analytical applications - on a data set in fast RAM, rather than on a slow disk.

As I was discussing SAP HANA with Vishal Sikka (SAP Chief Technology Officer and Executive Board member) and his team over the past months, I came to the conclusion that the software architecture embodied in HANA is a radical re-thinking of the assumptions underlying the enterprise software industry - and this could be transformative for the enterprise software industry and every industry it supports. Disruptive changes in speed and cost have always held the potential for transformations of industries, whether in transportation (from sailboats to airplanes), farming (from ox-driven plows to today's automated equipment), or mining (workers with pick axes to earthmovers and dynamite). As these industries transformed, they also led to transformations in the industries around them, and society as a whole. For example, fast, cheap, reliable transportation led to transformations of every industry from agriculture to energy to trade to government and even to war.

Vishal recently discussed a concept he calls "Timeless Software" (blog, video). Timeless Software embodies the notion that software must evolve as customer needs - and technologies available to satisfy them - change. Business processes and data need to survive even as the technologies around them get invented, flourish, and eventually passed by with new and (usually!) better successors. But what about the situation where the business needs change extremely rapidly, and the business can flourish or perish based on its ability to respond in real-time?

You could think of this scenario, where time is of the essence, as "timeful software" - scenarios in which you could transform an industry by eliminating latency - or lack - of information. HANA's speed allows batch processes to be performed more frequently, continuously, or transmuted into continuous processes. Can such speed - delivering information and insight into the hands of those who need it instantly when it is needed, or re-planning on an "as needed" basis rather than periodically - can such speed really transform an industry? Can moving information and deriving information in real-time make such a difference?

In many business processes, the answer is already, resoundingly "yes." Hotels check availability before confirming your reservation. Banks check for sufficient funds before cashing a check at the teller. Airplanes get rerouted and rescheduled when a volcano erupts in Iceland. But there are many other business processes which are executed periodically, in batches, today due to the cost and disruption to production systems. If the cost (performance) and disruption (latency, system unavailability windows) could be eliminated - as they can be with in-memory computing systems like SAP HANA - then the economics of businesses and industries could be substantially improved.

These "timeful" scenarios listed below are illustrative of those which I think will be enabled by SAP HANA, and which will lead to dramatic efficiencies, competitive shifts, and improved service, creating value for customers in such a way as to transform an industry.

[For the scenarios, please see the original blog at this link.]

Killer App

A killer app is a typically thought of as an application that is so beneficial that it drives widespread adoption of a new type of platform. This term was invented for the computer industry, with VisiCalc (driving the adoption of personal computers) being a canonical example. Once individuals, and businesses, adopted personal computers to run VisiCalc (or Lotus 1-2-3 for MS-DOS, or Excel and Word on Windows), users started using those same computers for many other applications, ranging from word processing to e-mail to web browsing. The impact of these second set of applications is more profound than was the impact of the spreadsheet, but it was the spreadsheet that paved the way for these applications by bringing PCs into mainstream adoption.

VisiCalc, Lotus, and Excel were really just containers that held data and applications ("macros"), and it was those applications that made the tools into killer apps, used for everything from budgeting to tax preparation to production planning to homework. In many ways, SAP BW is exactly analogous to a spreadsheet like VisiCalc or Lotus. BW is a container that can hold data and applications - applications including the lists of processes above. BW, with scenarios like the long lists above, will drive widespread adoption of in-memory computing (and SAP HANA, more specifically). Once HANA is in place as the database under SAP BW, customers will find many more ways to use HANA to transform their enterprises to much higher levels of performance, much as word processors, e-mail, and browsers are transforming business and society.

Will SAP HANA have the same impact as the PC? Will HANA be VisiCalc or Excel in my analogy? Time will tell, but time is exactly what SAP HANA gives you. And, perhaps in the end, time is the real "killer app."

Do you have additional scenarios to suggest for "timeful" transformations? Share them here!

Note: SAP is a former employer, and current client, of the author.

Is SAP HANA Right for You (Now)?

As mentioned in my previous post, SAP HANA has been the focus of much of SAP's technical and marketing communications of late, and has been of great interest to the influencer community (see these excellent posts by John Appleby (@applebyj) and Vijay Vijayasankar (@vijayasankarv) for example). Many SAP customers with whom I spoke recently are also interested in HANA for a number of reasons. Strategically, HANA is very important to SAP – it can deliver a substantial new revenue opportunity for SAP and its ecosystem, provide customers with game-changing benefits, and at the same time change the competitive market dynamics relative to Oracle in SAP's favor. Still, it is very early days for SAP HANA, and SAP's customers are not always willing to jump on a new technology until it has been proven to provide a real business benefit. How can you determine if SAP HANA is right for you (now)?


SAP HANA Benefits


Let’s start by taking a look at the benefits of SAP HANA. HANA can provide significant improvements in query performance, low latency between operational and analytical data stores, reduced administrative overhead, and both better TCO and price/performance ratios as compared to traditional data warehouses. Each of these bears some explanation; however, my explanation has run into several thousand words, so I will cover the details in my next blog.

Recently, SAP announced that SAP BW now runs on SAP HANA as a supported configuration, and the product is in a controlled release. SAP also announced an intention to deliver pre-built applications on SAP HANA, but these are not available yet. While I believe that BW on HANA is a "killer app" for HANA, others have urged SAP to focus on applications. According to Michael Krigsman, CEO of Asuret and an expert in achieving successful enterprise projects, "SAP's challenge is to humanize HANA, focusing on line of business use cases while de-emphasizing the underlying technology, at least from a marketing perspective. If line of business customers see huge value, they will demand that their IT departments purchase HANA."


Given the lack of availability of other packaged apps on HANA, I think the only real appeal of HANA can be – at this point – to SAP BW customers. Fortunately, given the benefits mentioned above (wicked fast query performance, reduced loading and preparation time, no latency between operational and analytical systems, and reduced complexity and cost), BW appears to be the perfect killer app for HANA – this will bring HANA in the door at many customer accounts, and then customers will find other uses for it.


Customers: Is SAP HANA Right for You (Now)


If you are a customer of SAP, chances are you will adopt HANA – maybe not today, maybe not tomorrow, but soon, and for … oh, wait, this is not that movie, and I don’t think you'll regret leaving your expensive and slow databases behind for HANA. Seriously, more than 13,000 SAP customers use BW today, and somewhere in the neighborhood of 35,000 customers use SAP Business Suite. In time, HANA will become so integrated with the Business Suite that customers will likely move to HANA, though this migration is some years out.


So, if you're planning on continuing to use SAP Business Suite, chances are you will adopt HANA. This means that the question for SAP customers is more "should I begin my adoption of SAP HANA now, or should I wait?" And the answer is quite simple: you should begin your adoption of HANA if there is a good business case for this adoption in your organization. After all, the price of a HANA appliance (assuming you already own your BW license but not the HANA software) is going to run you up to a couple of hundred thousand dollars. If you are not currently (or planning to start) using BW, then this might not be the right time for your enterprise to use HANA. However, a number of HANA-native applications are in the works from SAP, such as the famous CO-PA accelerator; as these applications come out, you may find a good business case for their use in your organization.


You should strongly consider using BW on HANA if your organization has:

• ... some reports or analytic applications that simply run too slowly to provide optimal business benefit, and a 5x to 1000x performance improvement would be material to your business,


• ... unacceptable downtimes, where your data warehouse is not available due to the need to load data,


• ... a need to see current, up-to-the-moment data, and/or
  


• ... a frequent need to create new reports or analytical applications, but IT struggles to maintain BW and keep it performing under changing user demands.

Note: even Business Warehouse Accelerator (BWA) customers may find that they have some reports or analyses that cannot be made to perform well in BWA, or there are long loading times, and users cannot see near-real-time data.


Many BW customers with whom I’ve spoken are happy with BW, but most have a few reports that run in hours instead of seconds, or IT can’t respond quickly enough to user requests, or the system is offline during the business day in one of their regions, or the data in the warehouse is "stale." If any of those issues sound like I’m describing your BW data warehouse, you might have a strong business case to move BW off Oracle (or whatever database you are using) and deploy it on HANA instead.


Even if you don’t think you need HANA (now), you could still benefit from HANA. Just go to your Oracle (or IBM, or Microsoft) sales rep and say that you are thinking about moving to HANA, and see if you can use that for leverage to get a better price on licenses and maintenance for your current database!


Systems Integrators: Is SAP HANA Right for You (Now)


SAP Systems Integrators range from IT mega-vendors like IBM and HP, to "pure play" consulting and integration companies like Wipro and Accenture, to boutique consultants like Bluefin Solutions and Jon Reed (@jonerp). The question these shops are all asking themselves right now is whether now is the right time to invest in developing SAP HANA skills. I can think of a single question that should help clarify the answer: does your firm do a lot of BW engagements?


If you answered "No" to this question, then this is a good time to keep an eye on HANA, but not a good time to invest in it. On the other hand, if you answered "Yes," then ask two more questions:

• Is SAP going to invest to make HANA important to my customers in the next twelve months?


• Can I develop HANA skills in-house or do I have to look outside?

Judging from the response HANA has been getting from SAP customers, and from the emphasis SAP senior management is placing on this technology, it is clear that SAP is going to invest to make HANA work, to develop more and more applications on the technology, and to drive the technology deep into the Business Suite. The marketing effort around HANA, combined with a working solution, is sure to drive a lot of customer interest in this technology.


A quick glance at the job boards shows that several consulting firms and integrators, SAP included, are looking outside for HANA skills. There are about 50 job posts for HANA skills on Dice.com and Monster.com, posted by firms like SAP, IBM, Fujitsu America, and PwC. The good news is that – if you already have BW (or even better, BWA) skills in-house, you can easily upgrade these skills to HANA. SAP is making it easier and easier to learn HANA, from the information and test software available on http://ExperienceSAPHANA.com/ to new and simple utilities like the Information Composer in HANA SPS3.


In fact, if you have an in-house data warehouse built on SAP BW, it should be a matter of days to migrate it to HANA, assuming you have the budget for a system (which could run between $50,000 and $300,000). Think about trying hard to get a test system from IBM, Cisco, Dell, HP, Fujitsu, or Lenovo. Also, think about splitting the cost of this system with your marketing department, because nothing should help you close your first HANA client faster than saying "we've done the migration for our own data warehouse, and let me share our experiences with you."


Independent Consultants: Is SAP HANA Right for You (Now)


Do you consult on data warehousing and/or BW? If so, you really should get started on HANA right now. The benefits of HANA are so compelling for BW customers, that you can really increase your business by demonstrating HANA to your clients.


Alternatively, if you are consulting in another area of SAP, and are looking to learn a little about HANA, you don’t have to go to the expense of buying your own HANA system. SAP has a HANA system running in the cloud, which you can access at http://ExperienceSAPHANA.com/, where you can study HANA and even implement your own systems to test out the software and develop your skills. Also check out this blog for information about another program SAP has created to get you hands-on with HANA.


ISVs: Is SAP HANA Right for You (Now)


Sadly, I don't think this is the right time for ISVs to think about developing custom applications on HANA. The tools are not really there, and the HANA platform team has other priorities right now. If your product uses BW, then by all means get some HANA experience, and think about how your application could evolve for a world of high performance analytics. If your solution includes BW content, then definitely begin testing HANA with your solution, so you can support your customers. But, unless you’re an SAP Demo Jam master, this may be too soon for you to adopt HANA for new, HANA-native applications.


Summary


Given the substantial performance improvements, up-to-the-moment data in the warehouse, reduced operating costs and complexity, and other significant benefits HANA offers to SAP BW customers, it seems likely that SAP HANA will have substantial appeal – and uptake – in the SAP customer base by the end of 2012. SAP BW customers should look at SAP HANA as the optimized database appliance for HANA, and most SAP BW customers should begin planning or piloting an adoption of HANA soon. SAP consultants and integrators should expect a great deal of demand for SAP HANA skills both in the short- and long-term, and should begin preparing now. It may be too early for ISV's to benefit from HANA, other than as it improves any BW components in the ISVs' solutions.

Note: SAP is a client, and former employer, of mine.

SAP HANA – the strategic context

This is part one of a three part series on SAP HANA.

Many years ago, SAP’s founders had the dream of implementing accounting and finances in real-time. They believed this could revolutionize business, making it possible for enterprises to have a clear picture of their financial positions at all time, enabling companies to make better decisions. Ultimately, this vision grew to include business processes across the enterprise, supporting real-time integration across all business processes in the enterprise. Over the years, SAP and other vendors have not always accomplished this level of real-time integration, but the days of batch processing of invoices, receipts, inventory updates, and other crucial enterprise information are largely behind us.

Except in business intelligence. Most enterprises today extract data periodically from their operational systems, transform that data into unified units and schema, cleanse the data of errors and gaps, aggregate the data to support faster queries, and then deploy that data into the enterprise data warehouse for reporting and analytics use. This process generally introduces a lag between when data are entered into the operational system, and when that same data are available in the data warehouse. This lag can be as short as a few minutes, or as long as a month, but is rarely less than an hour. Many enterprises “refresh” their data warehouse nightly (although when is “nightly” in a global enterprise?) or even weekly. One CRM system I used recently had a caveat on its reports, stating that “the data in this report may be 24 hours out of date.” In other words, on the last day of the quarter, a sales manager could not use the system’s reporting capabilities to determine if she had made her quota or still needed to make some more calls. For some applications, this lag time is unavoidable, but eliminating this gap between action and insight should be a goal of every IT organization.

For many enterprise topics, new ideas come from the consumer world – this trend is known as “the consumerization of IT” (or #CoIT, to insiders on Twitter). Consumer-facing companies (like Apple and Facebook) hear more clearly from their users and customers than Enterprise Software and Solutions companies (#EnSW on Twitter), and so they are often the source of innovation in the IT space. CoIT gives us many ideas about user experience (iPad!), social capabilities (Facebook!), mobility (iPad!), and scaling to massive data volumes (Facebook!). However, the consumer world does not offer us many good examples of real-time integration of operational and analytical data.

Into this critical need – powerful analytics on current data with real-time response – stepped SAP recently with SAP HANA. SAP aimed to bring the same real-time advantages to analytics that they brought to transactions. HANA is an extremely ambitious undertaking for SAP, which is not known for its leadership in the worlds of databases or analytics.

Over the years, SAP has offered its own database (SAP DB), which did not have a great deal of success in the market despite the obvious pricing advantages in comparison to commercial database products. Most notably, Oracle has been adopted by most large SAP customers, both for their operational databases and their data warehouses; Oracle has focused on the needs of large customers, and has achieved scalability, stability, and operational reliability not generally available from other commercial databases. Open Source databases have lagged far behind in these areas.

Additionally, SAP has offered first reporting and then generalized business intelligence solutions of its own (e.g., SAP BW), but these products have achieved only limited success, and that only in the SAP installed base. SAP BW has about 13,000 customers, but many of these customers use other analytical products alongside their SAP BW environments.

Recent years have seen SAP begin to make some serious moves to improve their position in the database and business intelligence spaces, specifically through the acquisition of Business Objects and Sybase. SAP has vaulted to a real leadership position in the BI world with the combination of its BW and “BOBJ” products, although it is still a distant #5 in the database market according to IDC.

IDC 2010 Database Market Share, Top 5 Vendors

Vendor	2010 Database Market Share
Oracle	45.2%
IBM	20.7
Microsoft	20.4
Teradata	3.3
SAP	3.0

If SAP could offer a discontinuous breakthrough, a game-changing technology, it might be able to capture a much larger share of this lucrative market, bringing some real benefits to the SAP shareholders and employees. Further, with Oracle's large share of the databases in SAP environments, a large increase in SAP’s share of this market would likely most hurt Oracle, SAP’s largest competitor in the applications business, reducing Oracle’s ability to fund its competing applications products through database profits, while simultaneously reducing Oracle’s insight into applications customers’ needs. Finally, if SAP could come up with a technology that provided real, new benefits to its customers, such as dramatic reductions in TCO, dramatic improvements in performance, or unification of the operational and analytical data stores for real-time data analysis, then SAP would be providing its customers with the kinds of benefits that could bring new levels of performance to their enterprises. This is precisely what SAP has set out to do with SAP HANA.

SAP HANA has no entered ramp-up, where SAP will take it from the first handful (or, technically, two handfuls) of customers up to several dozen, and then on to hundreds and thousands. Notably, SAP is using HANA internally, to speed insight for top management. At this point, HANA is primarily being used as a high-performance data store for BW, but stand-alone applications (such as “CO-PA Accelerator”) are not far behind, and eventually SAP plans to run their full suite of applications on HANA. Yes, that would mean a great potential savings for customers, and a significant reduction in business for Oracle, but this is years away from reality. In the meantime, SAP HANA looms as a potential boon to SAP shareholders, employees, customers, and partners.

The next blog in this series will discuss how to tell if SAP HANA is right for your organization – or for you.

Thanks to Mike Fauscette and IDC for providing market share data for this blog.

Please note: as of the time this blog was written, SAP is a current client of the author's.

SAP HANA Makes Progress and Threatens Oracle

SAP HANA has garnered a great deal of attention in recent weeks and months. For an overview of the technology and its potential, please check my recent blog on the subject, "The real (potential) impact of SAP HANA."

Since that blog was written, there has been a great deal of news in the SAP HANA world, and the surrounding cosmos as well. Rather than write another long blog on this topic, here is a list of some highlights:

• The pricing of SAP HANA is getting clearer. A low end (128GB to 256GB RAM) SAP HANA appliance could start at around $80K-$100K for the hardware. A few sources inside and outside the company (not under NDA) also indicated to me that the price of SAP HANA software is around $120K at the low end, stretching up to $1M to $2M at the high end. SAP says the pricing is not discountable, but early customers have told me that (at least for them), everything is negotiable. This puts the entry-level SAP HANA pricing at around $250K plus services. A pilot project could be done for less than $300K, with potential for very large business benefits. How hard is it to find a business analytics problem that is worth $300K if it can be completed 10x faster? For many businesses, there is no shortage of such analytics problems, and in some cases SAP HANA will actually perform 100x faster - or even better.

• Pricing for higher end SAP HANA boxes is not as clear. If your data can fit into 1TB of RAM, then these single-system boxes should be OK for you. But how will you know if your data can fit into that memory size? It seems (and I would welcome a knowledgeable person's correction on this if inaccurate) that SAP HANA can't be used when data can't fit entirely into one system's memory, which currently tops out at 2TB (and at much higher pricing - an additional $250K or so, from those few vendors offering that high-end configuration).

• How much data can fit on any SAP HANA configuration? From discussions with a number of current SAP HANA users, this is not always clear, and SAP has some work to do to deliver good sizing tools. On average, users I spoke with report compression ratios of between 4 and 10 times for data - that is, if the data would take up 400GB in an ASCII file, the same data with HANA's columnar compression would take between 40GB and 100GB in SAP HANA (your mileage may vary). Of course, the operating system and other software needs some memory to run, but this size of data should fit fine onto the smallest SAP HANA appliances. In a disk-resident database, you might need dramatically more disk space for such a data warehouse, since space will not be used efficiently (sparse data) and since the indices may take a significant amount of space - even more than the data sometimes!

• How about this for a sizing tool - load all your data on a removable hard drive (ENCRYPTED!!!), and bring it to IBM, HP, Cisco, Fujitsu, Dell, and anyone else offering a SAP HANA appliance. Have them load it up for you on their hardware, and test out your queries. If everything is as simple as some customers and integrators have been telling me (which is pretty close to how simple SAP has been saying it will be), then the hardware partners should be willing to do a pilot like this for free. You'll know exactly how much hardware you'll need, and you'll have a very good idea of its performance!

• As a killer new application, SAP HANA will be certified in November for running SAP BW. All BW reports, extractors, etc., should all run without changes (except MUCH FASTER) on SAP HANA after this release. Let's face it - the overwhelming majority of SAP BW data warehouses are running on Oracle databases. If you have an Oracle database license priced in the range of $800K (and many medium-sized data warehouses are on such licenses), with a 25% annual maintenance fee, you are talking about $200K per year just for the database maintenance. If you could migrate that database over to SAP HANA, you might find that you paid for it in the first year by dropping your Oracle maintenance, and at the same time you may be delivering ten times more users or ten times faster results on reports and analyses. It wouldn't be hard to imagine many Oracle databases being converted over to SAP HANA in the coming year - especially those running under SAP BW (and Business Objects soon too!).

• SAP recently also discussed several analytical applications that will be made available on SAP HANA. These are interesting, but not where the short-term value will be for most customers - this will be SAP BW or Business Objects running on SAP HANA, and ultimately the rest of the SAP Business Suite running on SAP HANA. Or Sybase, for that matter.

• Oracle may be feeling some heat from SAP HANA - Oracle just introduced a low end database appliance offering, with hardware and software bundled for between $100K and $300K. While it is unlikely that this appliance would be able to deliver comparable performance to SAP HANA for those applications most suited to HANA's current capabilities, this appliance would have broader application - for example, being currently suited to running production transactional applications such as SAP.

• At the high end, a SAP HANA appliance might cost on the order of $1M for hardware and $1M for software. An Oracle Exalogic data warehouse solution might cost 10x that number, delivering slower performance, consuming a lot more energy, and taking a lot more space.

• SAP has expanded the mechanisms for loading SAP HANA. You can now use log-based replication (replaying the logs on another database to copy it), and trigger-based replication (having a system notify SAP HANA when data is changed), as well as the previously available ETL-based replication (using BW extractors or other technology to copy data out in bulk/batch). The trigger-based replication mechanism is particularly interesting, because it updates the "data warehouse" (can you even call it that anymore with technology like this?!?) continuously, in near real-time. In addition, the trigger-based replication approach allows for consolidating data from several databases quickly and easily into a single data warehouse. If you have one database for your CRM system and another for your financials, both databases can propagate their updates into a single SAP HANA system for consolidated reporting and analytics. Trigger-based replication supports many-sources-to-many-targets replication.

• Business Objects metadata integration with SAP HANA metadata was not announced at SAP TechEd in Las Vegas, but perhaps will be available soon. Business Objects Explorer can run today on top of SAP HANA, which gives some level of integration that will benefit many SAP customers running SAP Business Objects. The more integration at the metadata level, however, the more appealing SAP HANA will become to Business Objects customers not running SAP systems.
  

Conclusions:

1. SAP HANA is getting to the point where any customer running SAP analytics (especially SAP NetWeaver Business Warehouse aka BW) should be starting or at least planning an evaluation.
2. SAP HANA will make its biggest impact - for customers - initially as an "accelerator" for SAP BW.
3. SAP HANA may pay for itself in reduced database maintenance payments ...
4. ... thus its biggest impact - for vendors - in creating pricing pressure on Oracle RDBMS for data warehouse instances.
5. We may be still a long way from seeing SAP HANA replace Oracle, IBM DB2, or Microsoft SQL Server as the transactional data manager for SAP Business Suite.

For more on the strategic impact of SAP HANA, please see http://www.enterpriseirregulars.com/39209/the-real-potential-impact-of-sap-hana/.

Disclosures:

I worked for SAP for six years, including the period when SAP originally acquired the technology at the heart of HANA. SAP is a client as of the time of this writing.

Salesforce.com Dreamforce 2011: Some insights from The Greatest (Cloud Computing) Show on Earth!

Ringling Brothers bills itself as "The Greatest Show on Earth," but Salesforce.com's Dreamforce 2011 could easily claim the title of "The Greatest Cloud Computing Show on Earth." With over 40,000 attendees, dozens of exhibitors, great concerts and parties, and presentations galore, the event was an unqualified success for Salesforce.com, but it also heralded a real coming of age of enterprise Cloud computing. Why?

1. The show was jam-packed with insight, guidance, case studies, education - and energy! Not only did this show demonstrate conclusively that Salesforce.com has vision (as usual), it also showed that Salesforce.com's ecosystem is moving ahead decisively into the Cloud, across many areas of their businesses. Sure, the keynotes were educational and entertaining, but the proof was elsewhere at the show ...

2. Salesforce.com knows how to throw a party, but they also know how to create a movement. The speakers across the board (from partners and customers) accurately and compellingly presented the Salesforce.com themes of the show - Cloud, mobile, and social. The Developer Zone was hopping with highly effective and productive sessions teaching developers how to become part of the Cloud, mobile, and social revolution - and what developer doesn't want that?

3. The Cloud Expo was hopping with businesses doing business. I visited several dozen ISV booths (like Kenandy, Google, Progress, Jitterbit, BMC Remedy, Informatica, Infor, LinkedIn, Workday, SnapLogic, Appirio, and Persistent Systems), and it was all I could do to get to a company representative. There was so much traffic, interest, business card exchanging, and follow-up meeting scheduling going on that I thought I was at an Oracle or SAP conference in 1993. Here are some photos from different attendees on Flickr that I found showing the mob scene on the show floor - but it wasn't just a mob, it was a mob determined to move their businesses into the Cloud:

http://www.flickr.com/photos/adriarichards/5241913777/
http://www.flickr.com/photos/islandgal/5247181522/
http://www.flickr.com/photos/mantala/6104550977/
http://www.flickr.com/photos/chuckschaeffer/6113830912/
http://www.flickr.com/photos/enzo6034/5250333129/

The vendors I spoke to said that they were very happy with their Salesforce.com relationship, that a very large fraction of their leads came from salespeople at Salesforce.com, and that the show was very much worth the money spent on the booth and related expenses. This is an ecosystem that has the Cloud, mobile, and social religion - because customers are leading the way.

Oh, and Salesforce.com didn't let this event go by without using it to maximize its impact on their top line. Deals were being struck, and you can expect to see the results in Salesforce.com's next quarter numbers.

4. The Salesforce.com team did a great job on the logistics for the show, despite its monumental size and enormous growth over previous shows. Keynotes were delivered well, camera angles were picked intelligently and for maximum visibility, schedules and meals were arranged to maximize traffic in the Cloud Expo, and thousands (seemingly) of developers were given the requisite skills to build their first Cloud, mobile, and social apps.

5. Salesforce.com has so much momentum in Cloud, mobile, and social, that several partners and even competitors arranged events around this show. Of course, there was the predictable Oracle attempts to do some guerilla marketing around the event - not Oracle's greatest and most successful marketing moments. One other competitor hosted an analyst gathering right after Dreamforce, and several partners had similar events before and after the conference.

6. Perhaps lost in all the hoopla were some impressive product announcements, demonstrating that Salesforce.com continues its ambition to lead by example, and to continue innovating to create customer value: Data.com, Chatter Connect, Chatter Approval, Database.com availability, social profiles, touch.salesforce.com, and much more.

Once again, Salesforce.com has put on a great show, but Dreamforce 2011 was much more than that - it may have marked the beginning of an industry-wide movement to the Cloud.

How vendors think about licenses: marginal price

CRN recently ran a story summarizing a Forrester Research study (sorry, I don't have the link - hopefully, someone will provide it in the comments) which revealed certain vendor practices which are stirring up resentment among CIOs. There is a lot within this report that our industry needs to take to heart, but there are a few items in there in which CIOs appear to want to have their cake and eat it too.

One item is based on a significant difference between how vendors view license fees for marginal users, versus customers viewing those fees (years after the contract is written) based on average price. Let's eavesdrop on a typical enterprise software sale to see the roots of this later dispute.

"So, I think I will need about 1000 users licensed for your CRM product this year, and then the population will grow by about 500 users next year, 1200 call center agents the year after, and then all international users in year four, which would be around 2500 additional users," said Carol Cio.

"I don't know if I'll still be on this account next year, let alone in four years," thought Sam Salesguy to himself. "Let's see if I can move this deal into this quarter."

"I'll tell you what," said Sam Salesguy, this time out loud. "How about if we do all 5200 licenses this year. If you can move that through procurement, I think can get you an additional 40% discount overall, and we'll phase in the maintenance on those licenses according to your schedule."

"I don't know if I'll still be in this job next year, let alone in four years," thought Carol Cio to herself, "but getting such a great concession from this vendor will improve my chances, make the CFO happier, make budgeting for the next few years easier, and will leave a little money on the side for us to bring that tablet roll-out for executives into this budget year."

"Let's do it," said Carol to Sam. "I'll push this through the CFO and procurement, as soon as you get approval from your sales management on the discount."

Later, Vince Veep said to Sam, "If we can pull in the deal this year, we can do the increased discount because we'll have reduced costs of administering contracts, we'll be sharing the risk with the customer, and because you and I will both be getting paid more for the deal." Vince thought to himself, "Of course this is what the company wants, because that's how they write our comp plans. If they wanted something else, they would compensate us differently. And it totally makes sense to share the risk with the customer, taking less money overall in exchange for a contractual commitment now by the customer. If the customer only wanted 1200 licenses now, they would pay more now for those licenses per user, and they would also pay more later for the other 4000 users. I would never offer this discount if the customer wanted an "out" clause, because then there is no shared risk. In fact, my friend Ed Economist would point out that the marginal price for the last 4000 users is lower than the unit price for the first 1200, so of course the maintenance price for those 4000 users is also lower. If the customer ever wanted to negotiate a cancellation of those last 4000 users, we'd have to go back to the higher price, plus we'd have to somehow be compensated for the commissions we already gave out and can't get back, plus also get compensated for the extra costs we have in contract administration and perhaps other areas of the company."

Perhaps I'm exaggerating what is going through Vince Veep's mind, but you get the point. To the customer, in economic terms, the marginal utility of a license in the future is about the same as the marginal utility of a license in the present, so that customer feels like she is getting a real bargain by getting that future license at a greatly reduced price, and often doesn't stop to think that there is a contract - that must be honored - underneath that price.

To the sales rep, the marginal utility of a deal in the future, when he may not have this account any more, is close to zero, so he would like to pull the largest check possible into this quarter, regardless of the interest of the vendor. The vendor's interest is enforced by a contracts and discount policies process, ensuring that the sales rep doesn't give away the farm, and that the customer gets a reasonable discount for taking on the risk of acquiring additional licenses.

If the customer needs to return those licenses in the future, the customer may be thinking about the average price of those licenses, but most likely the vendor is thinking about the marginal price of those licenses, plus a "restocking fee."

Obviously, in this short blog post, I'm oversimplifying a lot of things, and I'm not trying to fully represent the buyer's point of view, since that is widely covered elsewhere (e.g. at Constellation Research Group). I just want to communicate why it might be worthwhile to think about the marginal, rather than the average, price when thinking about license returns or exchanges. After all, understanding how your vendor is thinking might help you to better negotiate the best possible outcome for you and your enterprise.

The real (potential) impact of SAP HANA

Much has been written about SAP HANA. The technology has been variously described as "transformative" and "wacko." Well, which is it?

Disclosures
I have a few disclosures to make before I continue my analysis and comments on Hana:

1. I worked at SAP for six years, as well as eight years at Oracle (plus also at Ingres before that).
2. I was at SAP when the technology underlying HANA was acquired, though I am referring to and using no trade secrets or proprietary information in preparing this analysis.
3. I attended this year's SAPPHIRE conference in Orlando, and SAP paid for my airfare and hotel.

Relational Databases
Relational databases have dominated the commercial information processing world for twenty years or more. There are many good reasons for this success.

1. Relational databases are suitable for a broad range of applications.
2. Relational databases can enable access to data relatively efficiently even if the query was not initially envisioned when the database was designed.
3. Today's relational databases are economical, available on a broad range of hardware and operating systems, generally compatible across vendors, performant for many queries, scalable to fairly large data volumes without resorting to partitioning, suitable for partitioning when larger scale is required, based on open standards, mature, and stable.
4. There are a large number of developers, administrators, designers, and an ecosystem of service providers who are very knowledgeable about today's popular relational databases, and who are available at economic rates of pay.
   

NoSQL, Columnar, and In-Memeory Trend
There is an emerging trend towards databases that are designed to solve specific problems. While relational databases are good for solving many problems, it is easy to conceive of specific problems that are not well-solved by general-purpose databases. Relational databases are well-suited to handling structured data where the schema does not change, where text processing is not an important requirement, where data is measured in gigabytes rather than petabytes, where geographical or time-series (e.g., stream) processing is not required, and where the server does not need to support transactional and decision-support queries simultaneously.

Some problems do not fit those criteria. The data set is such that the schema varies from record to record, or over time. Text, image, "blob," or geographical data may be a dominant data type. More and more frequently, applications manage "big data," or huge volumes of data from millions of users or sensors. Some applications require simultaneous access to data for transactional updates as well as for aggregation in decision-support queries. For all of these cases, advanced architects and developers are looking at specialized data stores and data processing systems such as Hadoop, Cassandra, MongoDB, and others. These domain-specific data stores are known as "NoSQL" databases.

There is some controversy over whether NoSQL means "no SQL" or "Not Only SQL." Regardless, those non-relational stores such as Hadoop, are growing in popularity, but are not really a replacement for relational data stores. A key property of most commercial relational databases is their compliance with a principle called "ACID," which essentially guarantees that database transactions occur in a reliable way. Many NoSQL databases use techniques like "eventual consistency" to improve performance at the cost of inconsistent data - a sacrifice that is unsuitable for most business applications. After all, if you deposit money in a bank account, you want it to be available for withdrawal right away, not "eventually."

Another trend in the database world is towards new methods of storing data, without eliminating the ACID properties that business applications need, and without sacrificing the SQL language that is so well-known and widely supported. Two specific approaches are quite popular these days - columnar storage and in-memory databases.

Column stores, such as HP's Vertica or SAP Sybase IQ, store data by column. By contrast, traditional SQL databases store data as rows. The benefit of storing data as rows is that it is often the fastest way to look up a single value, such as salary, given a key value like the employee ID.

Columnar databases group data by column. Within a column, generally speaking, all the data is of the same type. A columnar store, therefore, stores data of a single type all together, which can give advantages such as the possibility for significant compression. Good compression can lead to reduced disk space requirements, memory requirements, and access times.

In-memory databases take advantage of two hardware trends: a significant reduction in the cost of RAM, and a significant increase in the amount of addressable memory in today's computers. It is possible, and economically feasible, to put an entire database in memory, for fast data management and query. Using columnar or other compression approaches, even larger data sets can be loaded entirely into main memory. With high-speed access to memory-resident data, more users can be supported on a single machine. Also, with an in-memory database, both transactional and decision-support queries can be supported on a single machine, meaning that there can be zero latency between data appearing in the system, and that data being available to decision-support applications; in a traditional set-up where data resides in the operational store, and then is extracted into a data warehouse for reporting and analysis, there is always a lag between data capture and its availability for data analysis.

SAP HANA
Several years ago, SAP acquired Transactions In Memory, a company that had developed an in-memory database. Over the years since, at virtually each annual SAPPHIRE conference, SAP has discussed how this in-memory technology would revolutionize business computing, but I personally found the explanations to be somewhat short on convincing details.

Even the name, HANA, has changed in meaning over the years. Initially, the name stood for "Hasso's New Architecture" (and a beautiful vacation spot in Maui, Hawaii) and referred only to the software. Today, HANA stands for High-Performance Analytical Appliance, and refers to the software and the hardware appliance on which it is shipped. In addition, HANA has evolved from a data warehousing database into a more general purpose platform.

SAP HANA does manage data in memory, for nearly incredible performance in some applications, but it also manages to persist that data on disk, making it suitable for analytical applications and transactional applications - simultaneously. But HANA's capabilities do not end there, and that may be the key to HANA's long-term value.



In the short-term, it seems that SAP still struggles to generate references for HANA, other than in a narrow set of custom data-warehouse-type analytics. That may obscure where HANA can really deliver its first market successes.

When HANA is generally available, it is expected to include both SQL and MDX interfaces, meaning that it can be easily dropped into Business Objects environments to dramatically improve performance. Some Business Objects analyses, whether in the Business Objects client or in Excel, can achieve orders of magnitude of performance improvement, with very little effort. Imagine reports that used to take a minute to run now running instantaneously. Imagine the satisfaction of your BOBJ user community if all or most of their reports and analysis ran instantaneously. Line-of-business users will pay for this capability, and that will open the door for SAP HANA in Business Objects accounts. After HANA gets in the door, I'm sure the CIO will find tons of additional uses for it. This is huge, and will generate truckloads of money for SAP, while also making customers super-satisfied.

And think of what SAP HANA means for competitive comparisons with Oracle, SAP's maximum enemy. Larry wants to sell you Exalogic and Exadata machines, costing millions; Hasso wants to sell you a simple, low-end, commodity device delivering the same benefits. If I were SAP, I'd have sales reps with HANA software installed on their laptops, demonstrating it at every customer interaction, and comparing it (favorably) with Oracle Exadata, and suggesting that customers demand that Oracle sales reps bring in an Exadata box on their next sales call - and not to bother showing up without one. Larry wants to sell you a cloud in a box; SAP will sell you apps on the cloud, or analytics in a box for hundreds or a thousand times lower cost than Oracle's solution.

The longer term benefits of HANA will require new software to be written - software that takes advantage of objects managed in main memory, and with logic pushed down into the HANA layer. I'll post more on this potential in the future, but just think of what instantaneous processing of enormous data sets will mean to business - continuous supply chain optimization, real-time pricing, automated and excellent customer service, and much more.

Summary
In the long run, SAP HANA may indeed revolutionize enterprise business applications, but that remains to be seen. Right now, SAP HANA should be capable of creating substantial customer benefits - and generating a very large revenue stream to SAP.

IT Jobs Recovery Continues, Picks Up Steam

There have been a spate of news stories in the IT press covering the state of IT hiring. Once a quarter or so, I look at the job posts in Dice.com across many different keywords - a useful proxy for the IT hiring market at least for the U. S.

Three months ago, IT hiring was definitely picking up across the board. Fast-growing skills included iPad, HTML5, Amazon, Android, Twitter, and Facebook; categories with the greatest number of jobs posted included SQL, Oracle, Java, Windows, Unix, and Linux. Versus last summer, there was about an increase of about 30% in the number of jobs posted. How have things changed in the past three months?

Generally, the jobs picture is very similar to three months ago, with an overall increase in the number of jobs posted of around 6.1% (46.2% vs last April). If these jobs are being filled, this shows a sustained and strong increase in demand for IT skills. Within the overall IT category, this analysis does not cover some areas separately (e.g., desktop support, systems administration, network administration or really many network skills, and database administration), although many of these categories get covered through the occurrence of the relevant keywords. Perhaps I'll add these topics for future analyses.

Overall

This analysis looks at four general areas skills for which IT hires: Database, Applications, Languages, and Platforms.

Database

The skills keywords I look at in the Database category include SQL, Oracle, SQL Server, PL/SQL, MySQL, DB2, Sybase, Hadoop, Informix, NoSQL, and many variants on these terms.

Across the board, there is significant increased demand for all these database skills versus three months ago (+5.1%), and versus a year ago (+34.5). Hadoop and NoSQL were not being tracked in the analyses from a year ago, but the fastest growth in other categories since last year and three months ago has been for MySQL (+52% vs last year, +9% vs 3 months ago). Hadoop and NoSQL demand has grown by 12% and 69% vs 3 months ago.

The weakest demand in this category has been for Informix (-12% vs 3 months ago, +10% since last year) and Sybase (+0.9% vs 3 months ago, -5% since last year).

Although there is rapid growth in demand for skills like Hadoop and NoSQL, the largest demand by far is for more standard skills like SQL, Oracle, and SQL Server.

Skill	Jobs	Growth over 3 months	Growth over 12 months
SQL	23453	5.6%	33.8%
Oracle	16684	4.7%	33.1%
SQL Server	9692	4.6%	34.7%
PL/SQL	3587	4.5%	35.1%
MySQL	3123	9.3%	51.5%
DB2	2564	4.8%	28.5%
Sybase	1314	0.9%	-4.7%
Hadoop	363	12.0%	New comparison
Other NoSQL	245	69.0%	New comparison
Informix	168	-12.0%	9.8%

Applications

The skills I look for in the Applications category include SAP, BASIS, ABAP, Dynamics, Peoplesoft, Siebel, eBusiness Suite, Salesforce.com, and related terms.

Three months ago, this was the category with the strongest recovery versus a year ago. Demand is still strong for applications skills, with the category growing by 4.5% over the past three months, and up 47.4% versus a year ago. The fastest growth in this category is clearly for SAP-related skills. Job posts mentioning SAP are up 8.3% vs three months ago and up 59% over the past year; ABAP and BASIS also had very strong growth, with job posts up around 70% for each skill over the past year. Salesforce.com job posts went up compared with last year (+37%), but were down over the past three months (-3.8%). Oracle's application suites (not including Fusion, for which I have no data) show much more modest growth or shrinkage, with Peoplesoft the lone bright spot (+54% over the past year). Microsoft Dynamics has also shown growth over the past year (+31%), but has declined vs three months ago (-2.9%).

The weakest demand in this category is for Oracle eBusiness Suite, for which demand has declined by 36% over the past three months.

The skills to know in this category, if you want to be employable, are SAP and Microsoft Dynamics, but the latter covers many different applications - SAP is where the job security lies. Of course, job security and overall compensation are not always aligned, but given the rapid growth in the SAP category we can expect very strong salaries there.

Skill	Jobs	Growth over 3 months	Growth over 12 months
SAP	7475	8.3%	58.6%
Dynamics	6653	-2.9%	31.3%
BASIS	3858	18.3%	67.3%
Peoplesoft	2475	6.7%	54.4%
Siebel	1305	-5.0%	13.2%
ABAP	815	1.5%	70.1%
Salesforce.com	714	-3.8%	37.0%
eBusiness	48	-36.0%	New comparison

Languages

The skills included in this category include Java, JavaScript, HTML, XML, C#, C++, Perl, AJAX, PHP, Python, Ruby, COBOL, Flash, Silverlight, HTML5, Assembler, PowerBuilder, and Fortran.

Demand for programming language skills surged, growing 58.5% vs a year ago and 7.4% over the past three months.

In particular, HTML5 demand really took off, growing 45.2% over the past three months and well over 1000% over the past year (from a very small base). There was such strong demand growth for all skills, that it is more useful in this category to speak about the area of weakest growth - Adobe Flash. Demand for Adobe Flash skills has grown by only 7% vs a year ago. Comparable skills - HTML5 and Microsoft Silverlight - have grown much faster.

Given the large number of jobs requiring Java skills, the growth in demand for Java by more than 33% over the past year is encouraging, and should indicate a recovery in compensation as well.

Skill	Jobs	Growth over 3 months	Growth over 12 months
Java	16152	5.5%	33.4%
HTML	9736	7.3%	57.1%
XML	9651	9.0%	27.0%
JavaScript	9618	5.4%	New comparison
C#	7940	9.7%	37.5%
C++	5987	4.0%	21.3%
Perl	4948	4.0%	25.5%
AJAX	4440	7.1%	36.9%
PHP	3022	10.6%	44.9%
Python	2601	15.8%	59.7%
Ruby	1540	15.8%	95.9%
Silverlight	982	12.6%	New comparison
COBOL	735	6.1%	36.9%
Flash	646	2.2%	7.0%
HTML5	540	45.2%	1488.2%
Assembler	212	12.2%	11.0%
PowerBuilder	155	26.0%	39.6%
FORTRAN	68	1.5%	30.8%

Platforms

Platform skills analyzed in this category include Windows, Unix, Linux, IBM, VMware, Open Source (new), Embedded, Mainframe, Android, Blackberry, Palm, iPhone, iOS, iPad, Azure, Amazon, Google, Yahoo, eBay, Twitter, Facebook, Mac, and Widget. I had an error in the way I was querying Mac jobs, so that has influenced some of the results.

Platform skills are also in great demand, with job posts up 5.8% vs three months ago and up 43.5% over last year. Again, there was very strong growth in demand across nearly all platform skills, with the fastest growth in iPad (-3.7% vs three months ago but up more than 3000% over the past year), Amazon (+12.8% over the past three months and +332% over the past year), and Android (+19.8% vs three months ago, and +270% vs last year).

Over the past year, demand has declined for only one skill (Widgets, -14%), but demand has softened over the past three months for Embedded Systems (-4.6%), iPad (surprisingly -3.5%), Palm (-6.9%), Widgets (-14.9%), and eBay (-3.3%).

Although all platforms are doing well, the top three (Windows, Unix, and Linux) are all showing strong demand and growth; it appears that Linux demand may surpass Unix demand very soon.

Skill	Jobs	Growth over 3 months	Growth over 12 months
Windows	12641	2.3%	28%
Unix	10692	3.9%	21%
Linux	10185	2.1%	32%
IBM	5850	7.4%	77%
VMWare	2609	2.8%	47%
Open Source	1955	New comparison	New comparison
Mainframe	1776	5.8%	32%
Embedded	1652	-4.6%	13%
Android	1170	19.8%	270%
Google	1132	27.0%	35%
Blackberry	1027	7.7%	77%
Amazon	1019	12.8%	332%
iPhone	1013	7.1%	162%
Mac	856	New comparison	New comparison
iOS	832	24.9%	New comparison
Twitter	683	7.4%	141%
Facebook	582	17.6%	135%
iPad	413	-3.5%	3077%
Yahoo	192	14.3%	16%
Palm	162	-6.9%	22%
Widget	149	-14.9%	-14%
Azure	103	80.7%	94%
eBay	88	-3.3%	42%

Surprising Findings

• Big declines in demand for Oracle eBusiness Suite and Oracle Siebel skills.
• Huge increases in demand for Microsoft Azure, SAP Sybase PowerBuilder, and SAP BASIS skills.
• Demand for Microsoft Silverlight developers surpasses demand for Adobe Flash developers!

If you have any skills you'd like to see tracked in these posts, please post a comment and I'll see about including them next time. Thanks!