Custom data format - `Deserializer::deserialize_str` implementation

Question

Link to playground

I am trying to implement a custom data format with serde, I've been struggling with the deserialize_str method

pub struct Deserializer<R> {
    rdr: R,
}

impl<'de, 'a, R: io::Read + 'de> de::Deserializer<'de> for &'a mut Deserializer<R> {
    fn deserialize_str<V>(self, visitor: V) -> Result<V::Value>
    where
        V: Visitor<'de>,
    {
        let len = self.read_i16()?; // implemention below
        if len == 0 || len == -1 {
            return visitor.visit_borrowed_str("");
        }
        let len = len as usize;
        let buf = self.read_exact(len)?; // implemention below
        let out_str = std::str::from_utf8(&buf)?;

        // visitor.visit_borrowed_str(out_str) doesn't compile
        visitor.visit_str(out_str) // compiles but errors
    }
}

impl<R: io::Read> Deserializer<R> {
    fn read_exact(&mut self, len: usize) -> Result<Vec<u8>> {
        let mut buf = vec![0; len];
        self.rdr.read_exact(&mut buf)?;
        Ok(buf)
    }

    fn read_i16(&mut self) -> io::Result<i8> {
        self.rdr.read_i16::<byteorder::NetworkEndian>()
    }
}

When using visitor.visit_borrowed_str(out_str), I get the error

    |
94  | impl<'de, 'a, R: io::Read + 'de> de::Deserializer<'de> for &'a mut Deserializer<R> {
    |      --- lifetime `'de` defined here
...
149 |         let out_str = std::str::from_utf8(&buf)?;
    |                                           ^^^^ borrowed value does not live long enough
150 |
151 |         visitor.visit_borrowed_str(out_str)
    |         ----------------------------------- argument requires that `buf` is borrowed for `'de`
152 |     }
    |     - `buf` dropped here while still borrowed

I understand that out_str needs to somehow live longer than its scope, but I can't find a way to go about it.

Answer 1

To use visit_borrowed_str, you need to hand it a reference to something that lives as long as your deserializer. Creating a new temporary Vec with read_exact won't do, you need to get access to the underlying slice, e.g. std::str::from_utf8(self.rdr.get_ref()[self.rdr.position()..][..len]) or similar. If you want to keep R a generic std::io::Read, I think you can't use visit_borrowed_str. serde_json e.g. handles this by having a special Read that returns a reference to the underlying data if it can, and then only uses visit_borrowed_str if it does have a reference to the underlying data.

Also, if you ask a deserializer to deserialize to a borrowed string when it can't, it must necessarily error. That holds true for serde_json as well. So the error from visit_str is not an error in your deserializer implementation, but an error in how you use the deserializer. You should have asked to deserialize to a String or Cow<str> instead (not that your serializer could ever give you a Cow::Borrowed, but asking for a &str just isn't a good idea with any deserializer, asking for a Cow<str> is the thing generally recommended instead).