In true Web 2.0 fashion, I’m attending the SAP TechEd event in Berlin by watching the virtual TechEd stream, with live coverage of the main keynotes and interviews in key areas. You can watch a replay yourself on the Virtual TechEd site.

The session started off with the new (excellent) “run better” SAP ad, featuring a wide selection of SAP customers:

Mark Yolton then stepped up on stage to welcome the crowd and members of the SAP Community Network. He emphasized the importance of people and community at the event — 15,000 attendees across the four locations, with over a thousand hours of lectures and on-hands sessions.

Next up was SAP CTO Vishal Sikka, who kicked off with a message designed to combat the positioning of various SAP competitors including Oracle: “People are looking for solutions, not stacks”. It’s now about the ecosystem, and it’s not possible for an ecosystem to come together on a single stack. Organizations need “non-disruptive” change. A stable core is important, but there has to be continuous evolution on top.

Vishal reiterated some of the messages from previous years, including the notion of “timeless software”, that allows you to innovate even with older platforms.

Vishal emphasized the continuing importance of SAP NetWeaver: “It’s the foundation of what we do”. He went over some of the numbers: 18,000 NetWeaver customers, 10,000 using BW, and 6,000 using the SAP Portal, many with more than 100,000 users, including the SAP community network itself with over 2 million members. “It’s alive, it’s kicking, and the numbers tell us that it’s not only being used to run the core infrastructure, but also extending it to their non-SAP applications. It’s strategic to our customers, and so it’s strategic to us.”

Vishal then invited Bjoern Goerke, in charge of the SAP NetWeaver platform, to discuss the new NetWeaver 7.3, due out later this quarter.

Bjoern explained how the NetWeaver team were delivering innovation without disruption into existing landscapes. Everything is now on the same release level, worked hard on maintainability, scalability, and performance – for example, reducing the downtime needed to install enhancement packs from several hours down to just a few minutes.

Next Vishal talked about the importance of Business Intelligence, and the upcoming release of SAP BusinessObjects 4.0 (project name “Aurora”). For the first time, all the clients tools have a common look and feel, and there’s improved integration with SAP Netweaver (identity management, BW, etc.).

The next subject was mobility. With Sybase, SAP is now the clear leader in enterprise mobility. There are now more than 4 billion mobile devices in the world, and Sybase is essential to using them. Over 2 billion messages are sent through Sybase systems every day (SMS, etc.), and Sybase recently crossed the threshold of a trillion messages through their servers. And Sybase also brings other great technology – their CEP (complex event processing) platform, the ASA and IQ databases will all be integrated into the NetWeaver framework.

He then invited Sam Yen on stage to show you how NetWeaver, BusinessObjects, and Sybase can work together.

The demonstration scenario was a series of Norwegian oil rigs, where there are sensors recording almost 600,000 events per minute, but there’s very low bandwidth between the rigs and the central offices, so  a lot of the processing has to be locally, and intelligently.

Suddenly, the oil production drops, and we can see the information in real-time using an Xcelsius (now BusinessObjects Dashboards) dashboard.

Let’s drill into the detail. We can see that Gate A was in the danger zone, and has been shut down.

What’s going on behind the scenes? Here’s the CEP development environment, that lets people easily create and change the real-time events that they would like to track and monitor

This is a very real need in oil and gas organizations today, but it takes a lot of painful programming. Because these systems are connected to NetWeaver, people can have these types of dashboards, running on top of Event Insight, as part of their enterprise portal.

Sam demonstrated the new SAP Enterprise Workspace, that users can customize with their own content, without any help needed from IT.

For example, he added the previous Event Insight dashboard, by choosing from the available applications…

And then simply dragging and dropping the module into the workspace.

Users have full ability to customize their experience including information from the outside world, etc.

Another example could be choosing vendors based on their carbon footprint. We look at the list of vendors:

And then he added a a sustainability module, using the functionality available in Carbon Impact.

Here’s the current version of Carbon impact shown within the portal:

Soon, SAP will be extending this functionality, with some of the products from the SAP BusinessObjects Innovation Center, such as Social Intelligence, in this case used in order to find sustainability experts inside my organization. People can easily do a search:

Find out how they are related to the rest of the organization:

And then invite them into the Enterprise Workspace.

Sam explained that the same portal and content can be made available for business partners outside the organization – Enterprise Workspaces in the cloud:

Vishal Sikka then moved from the improvements to existing products, such as NetWeaver, and BusinessObjects, to “New Horizons”, the new possibilities available in the future, with cloud computing, mobility, and in-memory computing: “On the one hand, we need continuous evolution. On the other hand, there are new possibilities – how to we bring fundamentally new solutions to the existing landscapes?”

Vishal outlined three big changes to technology possibilities:

(1) Cloud. There have been advances not only virtualization, but also in the ability to provision new services in public clouds. We need have the integrity of existing systems, AND improve the ease of consumption. How can we do that?

SAP want to bring the benefits of cloud computing to everyone, and the flagship product is Business ByDesign, which is not only a full suite for mid-size companies, but also the basis of the applications that SAP will deliver in the cloud.

Having a cloud platform lets us build applications, such as SAP StreamWork for collaboration and Carbon Impact, which runs on the Amazon Cloud.

(2) Mobile. Mobile internet use is already seven times the use of broadband. There are lots of new devices poised to join the market, including the RIM “PlayBook”. Users want a great experience. Vishal: “The screens want to be liberated from the tyranny of the desktop!”. What is possible?

There’s the Sybase Unwired platform that allows you to write applications once and deploy them to multiple platforms. And beyond the mobile devices, there are other platforms, such as SharePoint, Facebook, and Lotus Notes. The Gateway project was designed to make it easier to access SAP applications from these environments, as well as existing SAP systems back to R3 v 4.6C. It will be generally available later this year – you can already sign up for the ramp up program.

Microsoft and Cap Gemini recently worked together on an HR system for Microsoft that used the new Gateway functionality.

Representatives of the two companies explained that it enabled them to easily and quickly implement a new HR hiring application – something that is a critical application for Microsoft.

They explained that the application took about one-sixth of the time it normally would have done to implement, thanks to the Gateway functionality.

Sam Yen came back on stage to give a demonstration of the technology working together

He showed the end to end traceability of the SAP application within a SharePoint portal:

He explained that Gateway makes it easy to consume SAP data, no matter what front end you are using – for example, using a Macintosh.

Sam showed how you could create and start consuming SAP data on an iPad in less than two minutes. First, you choose the data you want to expose:

And in what form you would like to expose it:

Then you can use the standard Apple environment to compile an application:

And make it available to the iPad – in this case a simple employee search using information from the SAP HR database:

The last topic was in-memory computing, described by Vishal as a “once-in-a-lifetime opportunity”.

He explained that the current hardware possibilities were pretty much unimaginable ten years ago. With multi-core processing, software vendors have to rewrite their software to take advantage of parallelism, but once they have, the possibilities are amazing: real-time access to information, at a lower cost.

He gave an example of what’s possible, explaining that at a previous event in Berlin, in July, the CTO of one of the largest CPG companies in the world had challenged him to help them predict demand for their products at a row level – they had 460 billion records. Vishal put a team on it, and found that not only was it possible to access the data, it could be done 20x faster than anything the customer had previously tried. 460 billion records of data was about 45 terabytes. Compressed into a column database, it was around three to five terabytes. They put together a machine with ten server blades for around $530,000 using standard Intel parts.

The new HANA product is available for ramp up on November 30th, and he invited the audience to take part. One of the first customers is already in a pilot, running live. Vishal explained that the future direction is to “revitalize our entire product portfolio, starting with new applications – planning, forecasting, simulations”. He gave the example of utility companies using it to monitor and react to information from smart meters in real time.

Sam Yen came back on stage to give some examples.

The first was accessing over 4 billion rows of data in less than 0.03seconds, using HANA and the BusinessObjects Explorer front end.

The second example showed how you can get real-time access to live application data. He showed an SAP internal application for pipeline data. Here’s the initial view, seen on an iPad using BusinessObjects Explorer for iPad:

He then went into the SAP system to change some of the pipeline values, in a database containing half a million records. The data is then replicated to the HANA box in real-time using Sybase Replication services.

The simulation would normally take several hours to run, but now the data is reflected on the iPad in seconds:

Vishal finished with a high-level summary of the themes of his presentation: Innovation without disruption, through the new NetWeaver and BusinessObjects products, and the new possibilities afforded by the cloud, mobile use, and in-memory analytics.

Hasso Plattner, SAP’s co-founder, chairman, and “Chief Software Advisor” has been giving a series of talks including his keynote at SAPPHIRE 2009 on why in his view “disk has become yesterday’s tape”, and why column storage and in-memory techniques are the future for both data warehousing AND enterprise applications, displacing the 20-year reign of relational databases.

Experts have advocated column databases such as Sybase IQ and LucidDB for data warehousing for many years.

But now, based on research done at the his Institute (and a lifetime of trying to get the best performance possible for enterprise applications), Plattner is advocating basing both OLAP and OLTP systems on column storage in order to eliminate the need for the cumbersome ETL process, and reduce system complexity and the number of database tables.

More details are available in his white paper A Common Database Approach for OLAP and OLTP Using an In-Memory Column Database and a related presentation. The research was conducted using SAP’s TREX technology and real customer data (to see what the technology can do in a business intelligence context, check out SAP BusinessObjects Explorer).

Given that the vast majority of relational databases are used for OLTP transactions today, this would represent a radical change for the enterprise application market.

And although data warehousing will still need to exist, because of master-data management, data synchronization, data quality, etc., it would certainly be a lot simpler, and a lot faster, since much of it could be done directly on the “transactional” data source.

While nobody has promised that SAP will develop the concept as a product, Plattner has presented his views widely to SAP employees, and leaves no doubt that he thinks this is the way ahead.

Here’s the last slide of Hasso’s presentation at the recent SIGMOD 2009 conference for database researchers – this is  certainly something that we’ve been looking forward to for a while…