In 2006 Morantex made Beta 1 of Persistore available; the only .NET class library that provided a managed interface to Windows memory mapped files and shared memory on both 32-bit and 64-bit platforms. We received a significant amount of feedback including enquiries about future versions and requests for specific features; this feedback was reviewed and combined with existing product planning to define the major goals for a subsequent release, Beta 2 is that release.

Beta 2 utilizes a significantly enhanced kernel library, this library supports our proprietary read/write spin locks yielding improved performance in a multithreaded setting. The kernel also includes support for flexible high performance indexing using our proprietary relocatable dual-platform (32-bit and 64-bit) AVL tree technology.

Also in Beta 2 is a new high performance extensible buffer technology, this allows continuous dynamic buffer growth whilst avoiding the periodic expensive buffer copy and resize that is often relied upon in other designs. This allows large volumes of data to be continuously appended to SharedStream objects no matter how large that data may be.

These features along with many smaller additions and enhancements make Beta 2 a compelling and unique technology that promises to deliver unique capabilities and provide new options for designing systems that require both persistence and the ultimate in performance.

Many of the major new features form the basis of future LINQ support, enabling you to engineer software that leverages LINQ's impressive flexibility of data access in conjunction with very high performnace data persistence.

Morantex also felt that it was in the interests of both ourselves and our future customers, to ensure that Beta 2 fully supported Windows Vista and Windows Server 2008, the result is a powerful technology that can operate on a broad range of Windows platfroms both 32-bit and 64-bit.

Persistore will encourage the implementation of innovative solutions in many industry areas, some of those identified by early Beta 1 evaluators are

Securities
Delivering mission-critical processing of real-time data

The securities industry often faces significant challenges insofar as storing and manipulating large volumes of data in real-time. The need to have systems and applications keep up with high rates of information flow is non-trivial. In the past shared memory has often been the mechanism of choice for these kinds of problems, but was very inflexible and confined to a few specific aspects of a design.

Persistore can now provide easy to use out-of-the-box functionality for architects, designers and developers to exploit shared memory with ease. The technology aims to reduce to a minimum the overheads arising from interprocess communication and wasteful data copying, some of the major causes of latency in securities related processing.

Scientific Computing
Sophisticated concurrent processing of large data sets

Applying complex algorithms and mathematical analyses to large volumes of data has always been a challenge. By eliminating all explicit IO, Persistore eliminates the need to address storage and retreival of data as a separate processing step. Because vast volumes of data is to all intents and purposes accessed directly in-memory, Persistore provides scientists, engineers and designers with a powerful new means to manipulate data and to do so concurrently with ease.

Because all data is genuinely shared across processors and processes, it becomes possible to focus on the analytics issues leaving Persistore to transparently manage the storage and retrieval of data, be it managed objects or directly accessible primitive data types.

Search and Retreival
Acheiving acceptable results in acceptable time

With the proliferation of the worldwide web has come the need to store and retrieve large quantities of data, often in real-time. By eliminating the imbalance between file storage and memory based data manipulation, Persistore enables data to be stored and manipulated with no in-between conversion. Data may be indexed in memory and remain in memory along with those indexes, eliminating the unwieldy opening, closing, reading and writing of files. Massive volumes of data can simply be written to memory and manipulated in real-time leaving Persistore and Windows to worry about persistence.

Because Persistore incorporates a very efficient AVL balanced tree algorithm for indexes, it is straightfoward to create .NET classes that are decorated with index/key defintions and thus have Persistore automatically update any number of indexes whenever an object is persisted.

These indexes may then be used in several ways to facilitate the rapid location and retreival of the indexed data, furthemore an index may be structured (defined as being one or more fields within an object) or separate (arbitrary nature, with the key supplied separately) enabling a wealth of data storage and retreival scenarios to be supported.

What Next?

Morantex are readying Beta 2 for general release, once we have ensured the product meets our crieria for quality and stability on all supported platforms we will announce it's availability. You are encouraged to visit our website from time to time or to contact us either by registering on the website or sending us an e-mail, you will then be contacted as soon as the software becomes available for download.

Screen Shots

Here are a series of screen images that reveal some of the new features to be found in the Repository Explorer. The explorer is the primary tool used to perform administrative tasks on repositories.
   

Persistent repositories are yellow, volatile are blue. Green folders indicate .NET classes whilst yellow folders indicate a conventional data folder. When you start the explorer it will reveal every repository that is currently in use on that system, it will also dynamically detect repositories as they are loaded by applications.
Beta 2 supports the ability to visualize persisted data in various ways. This convenient feature reduces the effort required to investigate problems, or simply see at a glance the information stored. This example shows a SharedStream object being displayed as HTML, you may also use hex dump, plain text and several other visualisations.
This image shows data that has been accumulated by a simple web spider demonstration. The default pages of thousands of randomly visited sites were stored as SharedStream objects. These streams are derived from the .NET Stream so can be used wherever a Stream is. Note the red symbol indicating an active but not yet loaded repository.
The explorer support familar concepts like cut/paste, properties etc making a repository appear almost like a conventional disk. Folders may be created, deleted and renamed as can persisted objects and variables. Existing data files can simply be drag/dropped into the explorer main window, this causes them to be converted to persistent SharedStream objects.
Persistore stores .NET objects into a directory hierarchy that reflects the objects inheritance tree. This hierarchy includes directories that reflect the particular version of an object as well. This hierarchy is created automatically for any class, when objects are persisted for the first time.
The persistore kernel presents a uniform representation of a system's underlying hardware characteristics, at a glance one is able to see specific details that can aid in problem resolution and/or design choices.
Creating repositories is easy using the explorer. As can be seen one may create persistent (file backed) or volatile (not backed by any file) repositories using any of the supported options and features that are available to developers. A volatile repository permits very fast and sophisticated IPC capabilities to be incorporated in any application.
The explorer makes it straightforward to investigate suspected internal problems during development. If a bug in your application is suspected of damaging a repository's integrity, the validation feature can quickly pin down any corruption greatly aiding in debugging (this feature is also available programatically).