Aukio Data - Data storage and computing engine

Table of Contents

1. General

1.1. Source code

2. Vision / goal

Provide hackable, versioned, optimized, distributed, geometrical, arbitrary dimensional hypercube based data storage and computation engine (as inspired by the brain) for general purpose visual computing environment called Aukio.

3. Ideas / inspiration

3.1. Brain

3.2. CM-1 Connection Machine

https://en.wikipedia.org/wiki/Connection_Machine

Massively parallel (thousands of CPUs) connected via machine's internal 12-dimensional hypercube network allows to efficiently simulate arbitrary dimensional hypercube and network topology between computational units. So that when we are solving/simulating for example 5 dimensional problem, we can arrange computational units into virtual 5D network. See: http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html

we can pre-distribute data across computation units and perform parallel geometrical computation.

3.3. Integrate with lossless diamond square algorithm based multidimensional data compressor

  • Demonstrate using 3D UI for debugging algorithm internals.
  • Separate RGB/YUV channels into separate encoders/decoders
  • Make demo lossless image and sound CODECS.

3.4. Distributed computation and data storage

Lots of problems can be translated to geometry (use any shapes and as many dimensions as you need). Solution(s) to such problems could be then found via geometrical search/comparison/lookup results. As a bonus, such geometrical data storage AND computation can be naturally made in parallel and distributed.

3.5. The "Datacube" is a Fundamental Data Structure

The world is not just Sets (SQL), Hierarchies (XML/JSON), or Graphs (SPARQL). Arrays are the fourth pillar. Treating them as "just another file format" or "a blob in a table" is a fundamental error that leads to performance collapse.

4. Current status

  • More or less defined Vision / goal.
  • Collected some inspiring ideas.
  • Implemented very simple persistent key-value map.
    • Long term goal is to use it as a backing storage engine and implement more advanced features on top of this via layered architecture.

Created: 2026-09-20 su 00:35

Validate