React-query cache doesn't persist on page refresh

Question

I would like to set a 24 hours cache once a useQuery request has succeeded.

But as soon as I refresh the page, the cache is gone. I see it because I console.log a message each time the route is hit on my server.

How to prevent this behaviour and implement a real cache?

Here is the code:

   import { useQuery } from "react-query";
import { api } from "./config";

const _getUser = async () => {
  try {
const res = api.get("/get-user");
return res;
  } catch (err) {
return err;
  }
};

export const getUser = () => {
  const { data } = useQuery("contact", () => _getUser(), {
cacheTime: 1000 * 60 * 60 * 24,
  });
  return { user: data && data.data };
};


// then in the component:
  const { user } = getUser();

return (
<div >
  hello {user?.name}
</div>

I've also tried to replace cacheTime by staleTime.

Answer 1

React query has now an experimental feature for persisting stuff on localStorage. Nonetheless, I preferred using a custom hook, to make useQuery more robust and to persist stuff in localSTorage. Here is my custom hook:

import { isSameDay } from "date-fns";
import { useEffect, useRef } from "react";
import { useBeforeunload } from "react-beforeunload";
import { useQuery, useQueryClient } from "react-query";

import { store as reduxStore } from "../../redux/store/store";

const LOCAL_STORAGE_CACHE_EXPIRY_TIME = 1000 * 60 * 60 * 23; // 23h
const divider = "---$---";
const defaultOptions = {
    persist: true, // set to false not to cache stuff in localStorage
    useLocation: true, // this will add the current location pathname to the component, to make the query keys more specific. disable if the same component is used on different pages and needs the same data
    persistFor: LOCAL_STORAGE_CACHE_EXPIRY_TIME,
    invalidateAfterMidnight: false, // probably you want this to be true for charts where the dates are visible. will overwrite persistFor, setting expiry time to today midnight
    defaultTo: {},
};

const getLocalStorageCache = (dataId, invalidateAfterMidnight) => {
    const data = localStorage.getItem(dataId);
    if (!data) {
        return;
    }
    try {
        const parsedData = JSON.parse(data);
        const today = new Date();
        const expiryDate = new Date(Number(parsedData.expiryTime));
        const expired =
            today.getTime() - LOCAL_STORAGE_CACHE_EXPIRY_TIME >= expiryDate.getTime() ||
            (invalidateAfterMidnight && !isSameDay(today, expiryDate));

        if (expired || !parsedData?.data) {
            // don't bother removing the item from localStorage, since it will be saved again with the new expiry time and date when the component is unmounted or the user leaves the page
            return;
        }

        return parsedData.data;
    } catch (e) {
        console.log(`unable to parse local storage cache for ${dataId}`);
        return undefined;
    }
};

const saveToLocalStorage = (data, dataId) => {
    try {
        const wrapper = JSON.stringify({
            expiryTime: new Date().getTime() + LOCAL_STORAGE_CACHE_EXPIRY_TIME,
            data,
        });
        localStorage.setItem(dataId, wrapper);
    } catch (e) {
        console.log(
            `Unable to save data in localStorage for ${dataId}. Most probably there is a function in the payload, and JSON.stringify failed`,
            data,
            e
        );
    }
};

const clearOtherCustomersData = globalCustomerId => {
    // if we have data for other customers, delete it
    Object.keys(localStorage).forEach(key => {
        if (!key.includes(`preferences${divider}`)) {
            const customerIdFromCacheKey = key.split(divider)[1];
            if (customerIdFromCacheKey && customerIdFromCacheKey !== String(globalCustomerId)) {
                localStorage.removeItem(key);
            }
        }
    });
};

const customUseQuery = (queryKeys, getData, queryOptions) => {
    const options = { ...defaultOptions, ...queryOptions };
    const store = reduxStore.getState();
    const globalCustomerId = options.withRealCustomerId
        ? store.userDetails?.userDetails?.customerId
        : store.globalCustomerId.id;
    const queryClient = useQueryClient();
    const queryKey = Array.isArray(queryKeys)
        ? [...queryKeys, globalCustomerId]
        : [queryKeys, globalCustomerId];
    if (options.useLocation) {
        if (typeof queryKey[0] === "string") {
            queryKey[0] = `${queryKey[0]}--path--${window.location.pathname}`;
        } else {
            try {
                queryKey[0] = `${JSON.stringify(queryKey[0])}${window.location.pathname}`;
            } catch (e) {
                console.error(
                    "Unable to make query. Make sure you provide a string or array with first item string to useQuery",
                    e,
                );
            }
        }
    }
    const queryId = `${queryKey.slice(0, queryKey.length - 1).join()}${divider}${globalCustomerId}`;

    const placeholderData = useRef(
        options.persist
            ? getLocalStorageCache(queryId, options.invalidateAfterMidnight) ||
                  options.placeholderData
            : options.placeholderData,
    );
    const useCallback = useRef(false);
    const afterInvalidationCallback = useRef(null);
    const showRefetch = useRef(false);
    const onSuccess = freshData => {
        placeholderData.current = undefined;
        showRefetch.current = false;
        if (options.onSuccess) {
            options.onSuccess(freshData);
        }
        if (useCallback.current && afterInvalidationCallback.current) {
            afterInvalidationCallback.current(freshData);
            useCallback.current = false;
            afterInvalidationCallback.current = null;
        }

        if (options.persist) {
            if(globalCustomerId){
                saveToLocalStorage(freshData, queryId);
            }
        }
    };

    const data = useQuery(queryKey, getData, {
        ...options,
        placeholderData: placeholderData.current,
        onSuccess,
    });
    const save = () => {
        if (options.persist && data?.data) {
            saveToLocalStorage(data.data, queryId);
        }
    };

    // if there are other items in localStorage with the same name and a different customerId, delete them
    // to keep the localStorage clear
    useBeforeunload(() => clearOtherCustomersData(globalCustomerId));

    useEffect(() => {
        return save;
    }, []);

    const invalidateQuery = callBack => {
        if (callBack && typeof callBack === "function") {
            useCallback.current = true;
            afterInvalidationCallback.current = callBack;
        } else if (callBack) {
            console.error(
                "Non function provided to invalidateQuery. Make sure you provide a function or a falsy value, such as undefined, null, false or 0",
            );
        }
        showRefetch.current = true;
        queryClient.invalidateQueries(queryKey);
    };

    const updateQuery = callBackOrNewValue => {
        queryClient.setQueryData(queryKey, prev => {
            const updatedData =
                typeof callBackOrNewValue === "function"
                    ? callBackOrNewValue(prev)
                    : callBackOrNewValue;
            return updatedData;
        });
    };

    return {
        ...data,
        queryKey,
        invalidateQuery,
        data: data.data || options.defaultTo,
        updateQuery,
        isFetchingAfterCacheDataWasReturned:
            data.isFetching &&
            !placeholderData.current &&
            !data.isLoading &&
            showRefetch.current === true,
    };
};

export default customUseQuery;

Some things are specific to my project, like the customerId. I'm using onBeforeUnload to delete data not belonging to the current customer, but this project specific.

You don't need to copy paste all this, but I believe it's very handy to have a custom hook around useQuery, so you can increase its potential and do things like running a callback with fresh data after the previous data has been invalidated or returning the invalidateQuery/updateQuery functions, so you don't need to use useQueryClient when you want to invalidate/update a query.

Answer 2

Saving 16 bit unsigned image with PIL

In the code

img = Image.fromarray(pic, 'L')

The 'L' specifies 8-bit pixels, black and white according to the PIL documentation.

To create an unsinged int 16 bit image the 'I;16' option is required.

img = Image.fromarray(pic[0], 'I;16')

The stackoverflow post Read 16-bit PNG image file using Python says there is issues with this argument, however it is working fine for me using PIL ver 8.2.0 and Python 3.8.8.

other considerations

• You may also want to be careful with your data and noise array. They are unsigned 8 bit integers.

  data = np.zeros((t, h, w), dtype=np.uint8)
  noise = np.zeros((t, h, w), dtype=np.uint8)
  

  They can be converted to unsigned 16 using np.uint16 as the dtype parameter.

  data = np.zeros((t, h, w), dtype=np.uint16)
  noise = np.zeros((t, h, w), dtype=np.uint16)
  

• Is it possible for your processing to create negative numbers? Another issue could be caused when placing negative numbers into an unsigned integer array.

Answer 3

As this blog post suggests, you need help from an external library "libtiff". Since PIL struggles with 16-bit.

from libtiff import TIFF
tiff = TIFF.open('libtiff.tiff', mode='w')
tiff.write_image(ar)
tiff.close()