JFrog Artifactory: Partial file upload

Question

I'm working a Python 3 script to download data from a server in chunk of data (~1MB each, but can be setup externaly).

Each block downloaded must be uploaded to an JFrog Artifactory (JA) server (version 5.4.6, revision 50406900).

I'm using the HTTP Header 'Content-Range' to send the data blocks. But the JA is replacing the old data and keeping only the last block. The test file has 1164 byte and the header was sent right, with test block of 512 bytes (test only, no need big file to test it)!

- BLK#1: bytes 0-511/1164
- BLK#2: bytes 512-1023/1164
- BLK#3: bytes 1024-1163/1164

NOTE: Each PUT on JA was answered with a HTTP RC 201 (Created).

The syntax look all right (https://developer.mozilla.org/pt-BR/docs/Web/HTTP/Headers/Content-Range).

The first two blocks was 512 bytes long and the last one with 140 bytes. So we got the 1164 bytes of file.

I'm diging the official documentation, but until now I wasn't able to find a answer.

My questions are:

1) Doe's JFrog Artifactory able to receive partial uploads?

2) If so, how I can acomplish it?

Any help or hint will be helpfull!

Thanks,

WillianBR

Answer 1

Hi Folks,

I had figured out a way of solve the JFrog Artifactory annoying failure on partial upload of big files. I can't say it's "The one and best solution", but solve my issue and can be used with many scenarios.

I just created a inherited class of the io.RawIOBase, a base class of Python3. After a few tests I got the list of basic methods which I must implement.

class MemoryFile(io.RawIOBase):

It's working just fine! Right now I can't publish it here, because there is Company proprietary codes on it. But I'll clean up all and release a open source version. I'll publish on my GitHub and post the link here!

Just get the tast with a basic usage sample code:

# Each side must have a session to avoid issues
downReq = requests.Session()
# Here the request with all security tokens and required params (I'm ignoring SSL check for test)
response = downReq.get(urlTargetFile,stream=True, verify=False)
# My custom class take a Response as constructor argument
fpIn = MemoryFile(response)

# Another session to upload the date
upReq = requests.Session()
# Call the JFrog Artifactory address 
upResp = upReq.put(urlUp, data=fpIn , headers=headers, verify=False, stream=True)
# The SHA-256 hash to save into Audit Log
print("SHA-256: ", fp.hash())

I'll be back soner as possible with the full answer! The base classe and a 100% working example!