How Can I Hash By A Raw Pointer?

Question

I want to create a function that provides a two step write and commit, like so:

// Omitting locking for brevity 
struct States {
    commited_state: u64,
    // By reference is just a placeholder - I don't know how to do this
    pending_states: HashSet<i64>
}

impl States {
    fn read_dirty(&self) ->  {
        // Sum committed state and all non committed states
        self.commited_state + 
        pending_states.into_iter().fold(sum_all_values).unwrap_or(0)
    }

    fn read_committed(&self) {
       self.commited_state
    }
}

let state_container = States::default();


async fn update_state(state_container: States, new_state: i64) -> Future {
   // This is just pseudo code missing locking and such
   

   // I'd like to add a reference to new_state
   state_container.pending_states.insert(
      new_state
   )

   async move {
      // I would like to defer the commit
      // I add the state to the commited state
      state_container.commited_state =+ new_state;
      // Then remove it *by reference* from the pending states
      state_container.remove(new_state)
   }
}

I'd like to be in a situation where I can call it like so

let commit_handler = update_state(state_container, 3).await;

// Do some external transactional stuff
third_party_transactional_service(...)?

// Commit if the above line does not error
commit_handler.await;

The problem I have is that HashMaps and HashSets, hash values based of their value and not their actual reference - so I can't remove them by reference.

I appreciate this a bit of a long question, but I'm just trying to give a bit more context as to what I'm trying to do. I know that in a typical database you'd generally have an atomic counter to generate the transaction ID, but that feels a bit overkill when the pointer reference would be enough.

However, I don't want to get the pointer value using unsafe, because it just seems a bit off to do something relatively simple.

Answer 1

Values in rust don't have an identity like they do in other languages. You need to ascribe them an identity somehow. You've hit on two ways to do this in your question: an ID contained within the value, or the address of the value as a pointer.

Option 1: An ID contained in the value

It's trivial to have a usize ID with a static AtomicUsize (atomics have interior mutability).

use std::sync::atomic::{AtomicUsize, Ordering};

// No impl of clone/copy as we want these IDs to be unique.
#[derive(Debug, Hash, PartialEq, Eq)]
#[repr(transparent)]
pub struct OpaqueIdentifier(usize);

impl OpaqueIdentifier {
    pub fn new() -> Self {
        static COUNTER: AtomicUsize = AtomicUsize::new(0);
        
        Self(COUNTER.fetch_add(1, Ordering::Relaxed))
    }
    
    pub fn id(&self) -> usize {
        self.0
    }
}

Now your map key becomes usize, and you're done.

Having this be a separate type that doesn't implement Copy or Clone allows you to have a concept of an "owned unique ID" and then every type with one of these IDs is forced not to be Copy, and a Clone impl would require obtaining a new ID.

(You can use a different integer type than usize. I chose it semi-arbitrarily.)

Option 2: A pointer to the value

This is more challenging in Rust since values in Rust are movable by default. In order for this approach to be viable, you have to remove this capability by pinning.

To make this work, both of the following must be true:

• You pin the value you're using to provide identity, and
• The pinned value is !Unpin (otherwise pinning still allows moves!), which can be forced by adding a PhantomPinned member to the value's type.

Note that the pin contract is only upheld if the object remains pinned for its entire lifetime. To enforce this, your factory for such objects should only dispense pinned boxes.

This could complicate your API as you cannot obtain a mutable reference to a pinned value without unsafe. The pin documentation has examples of how to do this properly.

Assuming that you have done all of this, you can then use *const T as the key in your map (where T is the pinned type). Note that conversion to a pointer is safe -- it's conversion back to a reference that isn't. So you can just use some_pin_box.get_ref() as *const _ to obtain the pointer you'll use for lookup.

---

The pinned box approach comes with pretty significant drawbacks:

• All values being used to provide identity have to be allocated on the heap (unless using local pinning, which is unlikely to be ergonomic -- the pin! macro making this simpler is experimental).
• The implementation of the type providing identity has to accept self as a &Pin or &mut Pin, requiring unsafe code to mutate the contents.

In my opinion, it's not even a good semantic fit for the problem. "Location in memory" and "identity" are different things, and it's only kind of by accident that the former can sometimes be used to implement the latter. It's a bit silly that moving a value in memory would change its identity, no?

I'd just go with adding an ID to the value. This is a substantially more obvious pattern, and it has no serious drawbacks.