modify jpeg pixel data and keep all metadata intact

Question

I'm editing a JPEG image (blurring faces) and want to modify as little data as possible, namely I'm trying to keep the metadata as intact as possible. However, all the libraries for reading and writing image files I've tried so far leave some of the metadata behind.

Here's what I'm currently doing in Python (with Pillow):

import PIL.Image
import numpy as np

image = PIL.Image.open(input_fpath)

blurred_data = blur_image(np.array(image_data))
image.frombytes(blurred_data.tobytes())

image.save(output_fpath, quality="keep", sampling="keep", qtables="keep", **image.info)

Which loses at least the comment and JFIF blocks. I'm thinking that maybe I need to interact directly with libjpeg but maybe there's some tool or library that I'm missing.