Crux 'Oumuamua Assignment: Evict Transactions

earth pluto mercury neptune saturn jupiter comet

Introduction

This is the evict installment of the Crux tutorial.

We shall assume you have a Crux standalone node set up locally and are now comfortable with put operations and basic datalog queries. If this is not the case, you can find all you need to know on Earth, Pluto and Mercury. Click on a destination above to take you there.

Arrival

You arrive at the comet 'Oumuamua and pull along side, asking for permission to land. A voice comes over the communications system

How did you find us? Who sent you??

— Mysterious person

Choose your path:

"Kaarlang sent me" "I’m not sure how I got here, I found you by mistake"

You have permission to land

You are sent away. You must return to Jupiter and find Kaarlang.

comet

400

Space Port

You land on the space port and are ushered inside. The ships captain, Ilex, greets you.

Hello, it’s good to have you with us.

We are set to leave the solar system right away and as part of our service we offer people the right to be forgotten. Some are not worried that their information is kept here, however others want there to be no personal data left behind.

You may not have been told this yet, but this comet is actually a transportation vessel. It will take us to the star system Gilese 667C which is home to intelligent life far superior to our own. We all are hoping to find opportunities beyond our wildest dreams. All records of this transportation vessel and any life outside of the solar system are heavily monitored and wiped in the interest of preserving the normal technological advancement of the Human race. This means we know little of the beings we are going to meet.

Our task for you is to remove the records of the people who have chosen to be forgotten here.

— Ilex
Captain

You are excited by the prospect and agree to help. First you read the manual entry for evict as this will be the perfect tool.

Currently there are only four transaction operations in Crux: put, delete, cas and evict.

Transaction Description

put

Writes a version of a document

delete

Deletes a version of a document

cas

Writes new version of a document if the old version is as expected.

evict

Removes a document entirely

Evict:

Crux supports eviction of active and historical data to assist with technical compliance for information privacy regulations.

The main transaction log contains only hashes and is immutable. All document content is stored in a dedicated document log that can be evicted by compaction.

evict removes a document from Crux. The transaction history will be available, but all versions at the given valid-time or within the provided valid-time window are evicted.

Time in Crux is denoted #inst "yyyy-MM-ddThh:mm:ss". For example, 9:30 pm on January 2nd 1999 would be written: #inst "1999-01-02T21:30:00"

Along with the eid, put has 4 optional additional arguments:

start valid-time

(The time the doc will be evicted from.)

end valid-time

(The time the doc will be evicted until.)

keep-latest?

(Preserve the most recent version of the doc)

keep-earliest?

(Preserve the first version of the doc)

If no valid-time window is provided, the eviction will cover all time. keep-latest? and keep-earliest? defaults to false unless specified otherwise.

A complete evict transaction has the form:

[:crux.tx/evict eid valid-time-start valid-time-end keep-latest? keep-earliest?]

— Crux manual
Read More

You are given the data for the people who wish to be forgotten. You decide that you are going to embark on this adventure along with them so you add your name to the list.

(crux/submit-tx crux
                [[:crux.tx/put
                  {:crux.db/id :person/kaarlang
                   :full-name "Kaarlang"
                   :origin-planet "Mars"
                   :identity-tag :KA01299242093
                   :DOB #inst "2040-11-23"}]

                 [:crux.tx/put
                  {:crux.db/id :person/ilex
                   :full-name "Ilex Jefferson"
                   :origin-planet "Venus"
                   :identity-tag :IJ01222212454
                   :DOB #inst "2061-02-17"}]

                 [:crux.tx/put
                  {:crux.db/id :person/thadd
                   :full-name "Thad Christover"
                   :origin-moon "Titan"
                   :identity-tag :IJ01222212454
                   :DOB #inst "2101-01-01"}]

                 [:crux.tx/put
                  {:crux.db/id :person/johanna
                   :full-name "Johanna"
                   :origin-planet "Earth"
                   :identity-tag :JA012992129120
                   :DOB #inst "2090-12-07"}]])

Before you start the eviction process you make a query function so you can see the full results of anything stored in Crux:

(defn full-query
  [node]
  (crux/q
   (crux/db node)
   '{:find [id]
     :where [[e :crux.db/id id]]
     :full-results? true}))

You show the others the result:

  (full-query crux)

  => ([#:crux.query{:var id,
                    :value :person/kaarlang,
                    :doc
                    {:crux.db/id :person/kaarlang,
                     :full-name "Kaarlang",
                     :origin-planet "Mars",
                     :identity-tag :KA01299242093,
                     :DOB #inst "2040-11-23T00:00:00.000-00:00"}}]
      [#:crux.query{:var id,
                    :value :person/thadd,
                    :doc
                    {:crux.db/id :person/thadd,
                     :full-name "Thad Christover",
                     :origin-moon "Titan",
                     :identity-tag :IJ01222212454,
                     :DOB #inst "2101-01-01T00:00:00.000-00:00"}}]
      [#:crux.query{:var id,
                    :value :person/ilex,
                    :doc
                    {:crux.db/id :person/ilex,
                     :full-name "Ilex Jefferson",
                     :origin-planet "Venus",
                     :identity-tag :IJ01222212454,
                     :DOB #inst "2061-02-17T00:00:00.000-00:00"}}]
      [#:crux.query{:var id,
                    :value :person/johanna,
                    :doc
                    {:crux.db/id :person/johanna,
                     :full-name "Johanna",
                     :origin-planet "Earth",
                     :identity-tag :JA012992129120,
                     :DOB #inst "2090-12-07T00:00:00.000-00:00"}}])

The Crux manual said that the evict operation can take a number of arguments. You are an inquisitive person so you want to see the effect of a few combinations.

  (crux/submit-tx crux
                  [[:crux.tx/evict
                    :person/kaarlang] (1)
                   [:crux.tx/evict
                    :person/ilex
                    #inst "2000-01-01"] (2)
                   [:crux.tx/evict
                    :person/thadd
                    #inst "2000-01-01"
                    #inst "2500-01-01"] (3)
                   [:crux.tx/evict
                    :person/johanna
                    #inst "2000-01-01"
                    #inst "2500-01-01"
                    false
                    false (4)
                    ]])
  1. You try using evict with just the eid. [result]

  2. You try adding a valid-time to the eid. This gives the time from which the data will be evicted. [result]

  3. You add an end valid-time. This gives a date range that the documents will be evicted between. [result]

  4. You try evict with all arguments - an eid, start and end valid-time along with two Booleans: keep-latest? & keep-earliest?. [result]

You use your function and see that the transaction was a success.

  (full-query crux)
  => ()

All the data associated to any of the specified :crux.db/id has been removed from the Crux along with the eid itself.

The transaction history is immutable. This means the transactions will never be removed. You assure Ilex that the documents are completely removed from Crux, you can show this by looking at the history-descending information for each person.

You show the results for each person.

Results

1.

Using the eid alone results in the document being removed from the Crux from the time it was originally transacted until the evict transaction time. This operation has removed Kaarlang from Crux.

  (crux/history-descending (crux/db crux)
                           (crux/new-snapshot (crux/db crux))
                           :person/kaarlang)

  => ({:crux.db/id "efe634523d6867a3c6e4089074adf29b07b45f43",
       :crux.db/content-hash "4c7bc350e45e918c1dcac8ddfd881dd17c7c999d",
       :crux.db/valid-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-id 1599839177529345,
       :crux.db/doc nil})

2.

Adding a start valid-time means the relevant documents are removed from Crux from that time to the evict transaction time.

  (crux/history-descending (crux/db crux)
                           (crux/new-snapshot (crux/db crux))
                           :person/ilex)

  => ({:crux.db/id "f30c204cd61e131959fac45b4958e66ed642c7c7",
       :crux.db/content-hash "4fa7e81ff61cf588cb133f449139e6a2c268e371",
       :crux.db/valid-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-id 1599839177529345,
       :crux.db/doc nil})

3.

By using a valid-time range for evict you have removed all the documents within this range.

  (crux/history-descending (crux/db crux)
                           (crux/new-snapshot (crux/db crux))
                           :person/thadd)

  => ({:crux.db/id "05156e364f1d5b2d17d67f9ee80ad528c220e4a1",
       :crux.db/content-hash "b2aea825e89f3885309df9566b087c41b48d72a1",
       :crux.db/valid-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-id 1599839177529345,
       :crux.db/doc nil})

4.

Using the keep-latest? and keep-earliest? means you can decide if you keep some information in Crux. In this case you have chosen to not keep a trace and so your data is completely removed.

  (crux/history-descending (crux/db crux)
                           (crux/new-snapshot (crux/db crux))
                           :person/johanna)

  => ({:crux.db/id "cfdc197c1944e08658a0c890d1045d5216427a17",
       :crux.db/content-hash "7c024e441d51b7276315d746de13f3a4760aef60",
       :crux.db/valid-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-time #inst "2019-07-05T16:09:06.806-00:00",
       :crux.tx/tx-id 1599839177529345,
       :crux.db/doc nil})

Departure

The ship starts to shake as the engines are fired up.

Ilex thanks you and takes you to the Cryogenics department. You must be put into stasis as the journey will take around 25 years, even at the near light speeds of the ship.

You are astonished with the amount that you have done in one short week. How did little old you end up with an opportunity as big as this?

Your eyes get heavy as the cryogenicist initiates the hibernation process. As they do, you wonder if you’ll ever come back to the solar system.

ice

This is not The End

I hope you enjoyed learning about the basics of Crux. Although this is the final installment for the main tutorial series it is not the end of the Crux tutorial. Watch this space for more tutorial releases of the tutorial where you will be sent on assignments to solve more specific and complex tasks using Crux.

I’d love to hear any ideas for enhancements so don’t hesitate to get in touch.

Read more
crux   clojure  
Oct 24, 2019
Crux Development Diary Getting into Beta
jon
by Jon Pither
crux   clojure  
Sep 04, 2019
Introducing Crux JDBC Alternative to Kafka
jon
by Jon Pither