by David Thompson |

Capture 5D, World-Mirroring Digital Twins From Any Video Stream

Real-Time Speeds on Mini Computers

NO Lidar, Point Clouds, Sensors, RGB-D, OR Multi/Stereo Camera Setups Needed

Digital Twins are captured, stored and rendered in real-time. The software includes a separate, customizable, browser based, default app (GUI) that queries the software's data API. The extensible and customizable default GUI can render millions of Grassland data points at a time (like the game SimCity® or Civilization®). And, aside from rendering a 3D map of the entire world, also display data about people, objects and events as charts and graphs.

Rewind the hands of the GUI's 'clock' to move backwards and forwards through time to view the entire history of someone, some object or some place. External applications can query the network's API for purposes other than rendering visual simulations. This lets their internal systems scan any real-world environment from a perspective that's orthogonal to the four dimensions of space and time. In such a way that both physical and temporal occlusions are eliminated. Letting your systems analyze the real-world from a 5D perspective. In which all sides of every object and both the inside and the outside of every structure in multiple environments at multiple timestamps are visible to them at the same time. And in real-time. ( Above Video: data from the NYC Taxi and Limousine Commission. Taxi/limo geojson data is included in the node 'gui' directory for demo purposes).

Layperson's Description:

Grassland is peer-to-peer networked, AI software that takes video camera footage and remembers and stores the individual features, movements and geo-locations of every person, vehicle, building and relevant object as a virtual, 3D simulation in real time. The software shares only the relevant 3D data with the rest of the network so they can all have a complete simulation of what they're all seeing around the world.

It was originally designed because the Université de Montréal's AI lab[8] asked (April 13, 2018; ~3:20 PM) if I could help them solve the problem of giving AI an intuitive/experiential understanding of the laws of physics in the real world, due to some previous discoveries I'd made in this field[7]. The problem intrigued me so I went back to my home in Ottawa and a month later delivered a solution based on an "extension" of an overlooked mathematical theory; however, it was clear that it was better in every respect to turn the software into a public utility that was P2P networked not only because it would grow much faster and have greater adaptability (it can thrive and learn from censorship) but so that anyone could add cameras to it with no limits or restrictions and build apps that can query the network's data API for information. Letting any internet connected object "walk through" and "experience" people's entire lives (akin to how humans with eidetic memories experience life) or the "life" of any other object or building from all perspectives and timestamps at once without even needing to be there.

Although the data is all the same for everyone, it's just a model of the real world, anyone can build both public and private applications for people specifically based on what problems they want to solve using that data. It could be for finding lost children, helping a hedge fund model a retail store or factory's performance to predict quarterly earnings, giving an insurance company the tools to model and assess their risk portfolio or helping a city solve their traffic and emergency response problems. (See "Use Cases" below)

Technical Description:

Grassland is a P2P robotic vision and navigation system that is self-organizing, self-correcting and self-financing. The software efficiently scans any 2D video feed from any camera to generate a compressed, searchable, timestamped, real-time, 5D+ simulation of the world. The network's distributed API freely gives any machine complete situational awareness so that they can understand and trustlessly navigate any environment with no restrictions.

Grassland is a politically stateless and permissionless. Anyone can take part. Every node in the network has a permissionless and public API giving any external application or computer free access to Grassland data across the entire network, letting any internet connected object trustlessly internalize, understand and interact intuitively with both past and present states of the real world, digitally recreate or respond to even the tiniest changes taking place around the globe, from a butterfly flapping its wings in Calgary, to the lip-read conversations of pedestrians in Buenos Aires, to understanding that a motorcycle is signaling a left turn in Beijing all at zero cost and in real-time. While the combined work of the network makes it computationally intractable for nodes to submit fake data (see proof-of-work description below).

The network's game theory based dissemination system exhibits positive sensitivity to stressors; e.g. censorship makes it stronger. Letting it replay and learn from any attack or setback, including the source location and timing. Trustlessly learning the socioeconomic, domestic and cardiovascular (thermal cameras) nature of its adversaries via a prisoner's dilemma.


The three ('1','2','3') postulates/axioms below were considered to be "mathematically axiomatic". To accurately model a system like Grassland, in which the necessity for predictability cannot be overstated, it was sufficient to just regard each node (software instance) as an economic agent; human agency was not necessary since it becomes entirely subsequential as life gets enveloped, modelled and mirrored within. To put it simply, I took the following three postulates, and therefore the system they define, as "true" (as defined by "realism") on assumption only, then proceeded to deduce a programmatic theorem that naturally follows from, satisfies and is a logical consequence (not necessarily a causal connection) of those postulates--not quite 'features', but a 'logical framework'. The Grassland algorithm (and its underlying equations to be delivered in a follow-on supplement) is put foward as a proof of that theorem.
1. Trustless: There exists a computer networking system ("system" hereafter) wherein because successfully submitting fake data (see "Closed Under Computation") in the system approaches the limit of computational intractability, all of its nodes (or artificial economic "entities") find it more profitable to be honest.
2. Economic Incentive: There exists a system wherein as long as its entities are at least acting in their own economic self-interest the system would undergo continual expansion (in our case, the remaining "dark" areas of the map will be "lightened up").
3. Data Symmetry: There exists a system wherein no entity could maintain a data asymmetry (e.g. one-sided surveillance, stochastic (non-deterministic) outputs, etc.) so long as there are other entities at least acting in their own self interest. (A "scorched earth policy")
Closed[3] Under Computation:

It follows then that all necessary distinction between data that's valid or invalid, as defined by the system's utility is entirely "closed" under the system's proof-of-work. That is, nothing more than a "universally available method of computation", Δ, acting upon the network's federated data, Ε, is needed to determine Ε's validity or invalidity to the level of certainty that satisfies the requirements for utility, μ, tacitly "agreed" upon by the system's entities (since that's how we defined an entity above), such that the greater the total amount of computation, Δ, within the system the greater its capacity to validate Ε. And thus have no need of externalities not "closed under [its] computation" that require privileged access, specific locality, exclusive information, etc.

Therefore Δ is deterministic and is computable over any element of Ε within some reasonable amount of time, ψ, as determined by the requirements for utility, μ (and therefore a reasonable amount of computation). Such that, for any given ε ∈ Ε, Δ(ε) is the same for any of the system's entities that computes Δ(ε). And for any given ε ∈ Ε, Δ(ε) computed by any given entity of the system in time t, where t lies somewhere on the interval [0, ψ) and ψ is some small, positive real number. Moreover if t > ψ, and therefore not reasonable with respect to μ, none of the postulates could be satisfied since data symmetry could not be maintained or data validated within a practical or economically feasible timeframe. (In our case, speaking in a practical sense such determinism requires an implementation with the highest possible guarantee of consistent and expected behaviour between the entities because all entities must accept and reject the exact same data (binary sequences) and all within a certain amount of time.

Recursive Subjective Value Substitution via Entropy: It follows then that because the system commodifies and effectively discounts its socio-economic and behavioural data to zero, since it's no longer exclusive but ubiquitous, the 'economic incentive' left to each entity would therefore be the end of a subjective value substitution that constantly shifts away from the 'signified', the thermodynamic and Shannon entropy of its continual data generation, towards the only thing else that remains, its new, resultant 'sign'. With which, at every instance, the entity's entire subjective value must, by what its continued behaviour now suggests it to consistently act so as to increase, be completely, irreversibly, and recursively associated. Whose 'signified' is the irrefutably entropic instantiation of this artificially generated reward (as far as this system's underlying equations, which will be published in a follow-on supplement, are concerned, the data is, metaphorically speaking just a ubiquitously broadcasted "carrier signal" upon which the proof-of-work is encoded). To the extent that for every quanta/bit of information (certainty) gained for the entities of the system there is an associated, antecedent bit lost (or "anti-bit gained", so to speak) in entropy, whether they decode it as being Shannon or thermal.

Use Cases:

  Private data unneeded; the following can be done with data people continuously, consciously, and quite willfully display in public.
Capital has always migrated towards markets that more closely align with its ideal of "perfect information", since it can more effectively price the risk and protect its investment, irrespective of the personal preferences and location of the capitalist herself. While even having a single Grassland node is very useful, we attempt to show here that any community that adopts Grassland's innovation will see a disproportionate shift in global capital directed towards their socio-economic infrastructure, enjoy all the benefits of previously impossible innovation built on top of its API and all while being digitally and economically connected to other Grassland communities and individuals the world over. And where capital goes, humans must follow. (Note: the computer running the node software and the camera whose feed is being scanned don't have to be in the same location)
Insurance Companies, city planners, real estate developers/agents, home buyers, marketing firms, sales reps etc. This isn't just statistical information either; Grassland allows very specific analysis
Stock Analysts who use location based data (usually from wireless service providers) showing customer traffic and demographics to particular retail locations or computer vision on long-range photographs of manufacturing facilities to predict a company's quarterly earnings.
Economists, sociologists, historians, anthropologists, kinesiologists, epidemiologists, etc. are given an unvarnished and indelible depiction of people's lives in 3D from all viewpoints and timestamps without being restricted by either. And unlike social media or the news, it shows who people really are and not just the person they want others to see.
Local Businesses: A person about to buy a gas station can know precisely how many cars go there a day and how much they bought estimated from how long they spent pumping and the size of the tank from the make and model
Disease Control: An epidemiologist who needs to find people who might have come in contact with an infected person can see everyone whose timestamped tracklets[4] puts them in contact with a (potential) host's timestamped tracklet. Allowing them to virtually track, tag‡, and locate potential carriers and prevent further outbreaks.
Environmental Protection: At a macro level, it allows tracking of visual, salient pollution indicators as well as sources of pollution. On a micro level, it would allow a wildlife preserve to track a poacher before the poacher has even left their house
Impartial Accountability: A citizen or public servant who wants to know what any citizen or (rival) public servant (their extended family, friends, current and former romantic interests, business associates etc.) is doing (located, talking about, looking at, etc.) at this moment or has ever done before.* Affording them trustless, decentralised and innately financed undersight and sousveillance[5].
Prohibits Propaganda/False Information: For the physical world, it provides impartial verification of current or past events and pseudo-reality AI's like Deep Fakes, an embargo on fake news, unaccountability and ignorance (in any Grassland node's corner of the world, at least) by providing an authentication for reality† capable of being verified solely mathematically without the need for a trusted, human presence and whose validity increases with the density of nodes within an area.
Socioeconomic Data: The first node developed has been able to determine my neighbourhood's individual heights, estimated weights, walking gait identification patterns, estimated salary based on car model detection, family structure, daily schedules, and the sentiment of public conversations (via lip-reading) of various households. E.G. It can show on what days in July a resident mowed their lawn after getting it resodded, the pattern they mowed it in the eighth time and that it was the same day their neighbour had 5 guests over for a get together. It can be rewound and replayed from every angle in 3D. All of which can be converted to various tabulated data formats. This was through just one node scanning one digital camera feed.
Universal State Machine: Every person and every internet connected object will be able to "remember" every individual and event no matter where it is. This lets them not only internalize events regardless of distance, but also step into someone's experience and see what life is like currently through their "eyes". While giving AI's the ability to see through everyone's "eyes" all at the same time.
Logical Religious Substitute: Gives theists an all-seeing, all-knowing intelligence that's both everywhere and nowhere, watches over them, keeps and account of all their deeds in a ledger, rewards them with an incorruptible reward (on a reliable schedule and precisely as promised) if they memorize its word (private key), "hears" their prayers (reads their lips), and saves their lives (to disk) for "eternity"
Secure Parallel Multiagent Ideological Search: Grassland's instant accountability and data symmetry (by mutual economic self-interest) mitigates the negative effects of destructive dogmas that could be generated from the automated introduction of billions of new, customized normatives through AI generated narratives[7]. This allows a protected, programmatic acceleration of the reduction of cultural vulnerabilities and exploits inherent in ideological homogeneity by allowing multiple, receptive human agents in multiagent systems to safely conduct parallelized searches for new local-optima that outcompete the increasingly obsolete norms by which their cultures process data; creating flexibility in society's ability to solve never before seen problems while also providing the safety net of a cryptographically secure "reality".
* This use case will never be entirely precise for times when that person is in a place where there is an expectation of privacy. But you can make a lot of educated guesses with the right statistical models. Some of the more detailed, data science information will be added to the network's official model as the detail and computational requirements increase as per the schedule below, divided into "eons".
And there will always be people for whom Grassland is not a deterrent to abhorrent behaviour. But since they must therefore be extraordinarily irrational, they are by definition beyond predictability with or without Grassland.
If the corollary to Upton Sinclair's observation, “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” is as good a heuristic as the quote itself and if one can honestly say that Grassland provides both an authentic and logically testable "reality" as well as a source of income for all node operators, then one could also say that the former provides a useful tool to that rational "part" of the mind while the latter functions as a happily accepted "harness" for that obstinate, irrational part, which is so easily tricked, and so a number of its biases might, therefore placably and greedily allow themselves to be hemmed in and bribed in service to its rational counterpart.
Most manual data can't be included in blocks since it's "invalid" with respect to the set we refer to as "Closed Under Computation".
Because each node's transaction validation system uses an intentionally limited scripting language (necessarily easy to reason about, debug and foresee dangerous vulnerabilities) which takes raw binary values as input while at the same time its geographic coordinate system encodes everything from entire regions to extremely precise coordinates as single, fixed length, binary values this opens up the possibility of allowing very interesting logic, such as letting users create ongoing “bounties” which preallocate from their digital wallets a bonus payout of an amount they specify to all nodes whose camera's line-of-sight is pointed at, and thus gathering information from, a locale the user has marked in the GUI. The dimensions of this locale can range from half a cm2 on the earth’s surface to as big as a hemisphere. The size of the bounty itself would also be displayed to all users via the GUI.

Software Features:


Proof-of-Work Diagram

The results of each object detection on a frame is published along with hashes from different stages in the neural network.
Since it's astronomically improbable to have hashed those hidden activations into the correct digest unless those hidden layers were actually computed, this is a very useful proof-of-work.



[1]. We're not affiliated with the owners of these software titles. They're owned by EA Games, Firaxis and Microsoft, respectively.

[2]. Nash Equilibrium

[3]. Closure

[4]. Tracklet

[5]. Sousveillance

[6]."world-mirroring" is much terser than my own verbose description. It was first introduced to me by someone in an online forum. Though Gelernter's "Mirror World" ideas did not inform Grassland's design since I was not made aware of him until that moment, which was nearly a year after I had started writing the software. I've chosen to remain uninformed in regards to their details or implementation.

[7]. Deep Schizophrenia

[8]. Mila