Columnar Database and In-memory Processing

Columnar Database, precisely as its name, means a database that primarily stores and processes data by columns, rather than by rows. Most of the traditional relational databases and OLTP (Online Transaction Processing) cubes store and access data by rows. Due to the increasing challenges brought by today’s Big Data problems, which require ever faster processing of huge amount of non-relational data, columnar database is referred more frequently in present-day database technology discussions.

The major advantages of columnar database are typically twofold: First, it offers more efficient compression of the data in storage blocks, which results in overall less storage space and smaller amount of data to load onto disk or memory. Second, due to both the compression and self-indexing nature of the columnar data, it can drastically reduce disk I/O requirements and offer much better query performance, especially for those column-oriented operations such as SUM or COUNT. If the query operations are performed on limited columns only for a large data set with millions or trillions of records, columnar database usually offers notable performance improvement. Today’s well-known commercial products of columnar database include HP Vertica and Amazon Redshift (in AWS offerings), etc.

With memory chip technologies keep improving, RAM price dropping and non-volatile memory technologies readily available at present, in-memory processing or in-memory database (IMDB) becomes naturally affordable and popular. When the entire database or large blocks of data are stored and directly processed in memory, without disk I/O overheads, query performance can be significantly faster and predictable. Under such scenarios, the performance differences of row-based vs. column-based processing for a standard-size database may be less of a concern. Many enterprise database providers are exploring in-memory processing as their fast DB or Big Data solutions today. For example, in-memory options in Microsoft SQL Server 2014, IBM Informix, Oracle RDBMS, etc.

In the coming cloud world, distributed data stores will become more common. When huge amount and large variety of data need to be accessed across various storage units and processed at the same time with hybrid processing nodes, smart optimizations on storage, retrieval mechanisms and query performance may still demand careful considerations. We will likely see more data technology innovations and solutions with more intelligent designs and optimization algorithms built in.

Virtual Reality or Parallel Reality?

If ever possible, we all crave for expanded realities beyond the plain physical world that is experienced only through our limited basic senses.

From motion detection, eyeball tracking, to an instant fantasy world by putting on some 3D goggle or headset, human-interacting media reality or virtual reality (VR) are definitely joining the new technology fanfare nowadays. Whether it is Google’s expensive Google Glass, Oculus Rift VR headset (Oculus was purchased by Facebook in 2014), Razer’s OSVR headset, or the coming Sony, Samsung and many other vendors’ new VR gadgets, the current-day implementations of the VR are to distort our brain to accept the existence of the virtual world and the virtual connections with the visual contents presented – as if we are living and reacting at the same time with another place outside our immediate physical, where we seem to be, see, hear and touch with the objects and surroundings presented in the media or game. However in such scenarios, our brain always knows in advance that these are pure “virtual” and not real.

Star War Hologram Jedi Meetings
Star War Hologram Jedi Meetings
TriStrategist thinks that the frontier of the VR technologies will well be moved outside content visualization and gaming soon enough. In fact some of the best VR ideas we hope to see in the future and have imagined so far are again already in the sci-fi movies. Those Star War Jedi meetings had great communication channels where everyone can be called upon in real time through holograms no matter where their physical bodies are traveling in the universe. The realization of such VR technologies may not be far at all, just as we have to believe that humans could definitely colonize other planets with the advancements of technologies in the not-so-distant future.

To expand our actual reality, one way is to create another virtual or fake “reality”, then trick our brain to believe it and gather our basic senses around it. However, through further studies and advancements of modern physics, cosmology, biology, neuroscience, psychology, and of course aided by future technologies, we may well discover the existence of real ultra worlds which are yet to be detected or proved today. The parallel universe and wormhole theory could be the start, but the existence of other parallel realities could also be valid although they are still beyond our scientific understanding or even imagination today. Nature has vast unknowns waiting to be explored by us which could fundamentally change our concepts about space, time, energy, the power of our brain and undefined senses. If one day the parallel universe or parallel reality is proved true, we will be thrilled to no end. As we open our minds, seek and believe, the possibilities will be truly endless.

2015, A Year of Continued Transformation

The world is changing rapidly and we are living in an age of major transformations. For personal or for business, embracing the changes, looking forward to the future, being adaptive and flexible will become more important than ever. It’s certainly easy to say than done.

Almost all business leaders today agree that the coming years will see tremendous technology-based business transformations. The forces and momentum for changes have already been established in the broad market and society. Many of these transformations for businesses are taking place at this moment and year 2015 will surely be a significant year along the path.

Decade-old business models, mindsets or business processes will continue to be challenged and put under scrutiny as new technology innovations and new business concepts on the global scale are shaking up the society in every way. New breakthroughs will open ones’ minds and imaginations to far greater possibilities. Today’s technologies also helps enable many of the new business ideas to penetrate into the worldwide mainstream almost instantly.

When we look into the future, many seeds have already been sowed today. In 2015, TriStrategist thinks that we are likely to see fast changes in the following business areas, just to name a few:

– Cloud business: When IaaS are moving more towards commodity services, SaaS may become the differentiators in public cloud offering. Customers are seeking new features, flexibility and easy-to-understand pricing models in SaaS offerings.

– Device business: Worldwide competitions are only getting more and more fierce. It demands innovative ideas in manufacturing, selling and distribution, marketing, pricing and many more. Joint design and investment model will become a norm in device business as any new device comes and goes so quickly. Order-on-demand will likely be the preferred mode of operation for OEMs/ODMs and retailers. Speed and superior design innovations will be essential in all device business.

– Enterprise IT: Carried by the cloud computing waves, internally IT departments will likely move more towards SLA-based offerings – measurable on-demand or shared services models for more efficient and cost-effective internal infrastructure, platform and application support. Pain will be felt as many past established IT processes and roles will be shuffled through such changes.

– Ubiquitous Connectivity (UC): New gadgets, new sensors will continue to mushroom. UC will start taking clearer shape.

It will be an exciting time for many new entrants, but for large traditional businesses, trials and tribulations await because majority of technology innovations today are distinctively disruptive in nature. Yes, an elephant can dance, but for how long and how well is a serious question in today’s environment where new rivals and threats come from every corner of the world, possibly in the most unexpected manners.

For both personal and business, successful transformations will eventually come from the ready minds with visions, courage, dedications and agility. Peter Drucker once warned that yesterday’s breadwinner, “soon becomes a bar to the introduction and success of tomorrow’s breadwinner. One should, therefore, abandon yesterday’s breadwinner before one really wants to, let alone before one has to.” He also reminded us, “Do not kill tomorrow’s breadwinner on today’s altar”. If we have followed his wisdom and practiced routinely, we should hold onto the belief that successful transformation will be with us when we need it in the forward-looking new reality.