Uploading and Managing Files
Uploading a file to dStorage
./zbox upload command is used to upload a file on dStorage. Parameters for uploading a file to dStorage can be viewed by using the ./zbox upload --help command.
Here are the Parameters :
allocation
yes
allocation id, sender must be allocation owner
string
commit
no
save metadata to blockchain
false
boolean
encrypt
no
encrypt file before upload
false
boolean
localpath
yes
local path of the file to upload
file path
remotepath
yes
remote path to upload file to, use to access file later
string
thumbnailpath
no
local path of thumbnail
file path
Example :
Let's upload an info.txt file to allocation, the local path for the file is mapped to a local variable and the remote directory is mapped to a remote variable which is myfiles/info.txt.
Sample Command:
Response:
Verify whether the uploaded file is available on dStorage using list command required parameters are allocation ID and remote path
Response:
The response yields a lookuphash which works as a unique identifier for the uploaded file. Since you uploaded the file, the download payer section will show it as owner.
Copy and mark your lookup hash to a variable as it will be used for performing other file operations
export LOOKUPHASH=f15383a1130bd2fae1e52a7a15c432269eeb7def555f1f8b9b9a28bd9611362c
Uploading an Encrypted File to dStorage
Parameters for encrypting and uploading a file to dStorage are almost the same as uploading a file on dStorage and only requires an additional --encrypt flag with the command.
For instance if we want to encrypt and upload a sample info.txt file the command would be:
Command:
Response:
To verify whether the file is encrypted and uploaded to dstorage use the list command :
Response:
Scroll right in the response and check the IS ENCRYPTED section, it will say YES if the file is encrypted.
Updating the contents of a file on dStorage
Use ./zbox update command to update the content of an existing file in the remote path. Like upload command. Only the owner of the allocation or a collaborator can update a file. To add collaborators to an allocation, check File Collaboration.
Parameters for updating the contents of the file on dStorage can be seen by typing ./zbox update --help
Here are the parameters:
allocation
yes
allocation id
string
encrypt
no
encrypt file before upload
false
boolean
localpath
yes
local file to upload
file path
remotepath
yes
remote file to upload
string
thumbnailpath
no
local fumbnail file to upload
file path
commit
no
save meta data to blockchain
false
boolean
Sample Command:
Verify the contents of updated file on dStorage using the list command.
Sample Response:
Create Directory
./zbox createdir command is used to create directory on allocation for storing files.
--allocation
Provide Allocation ID
string
--dirname
Provide Directory Name and absolute path .
string
--h,--help
help for createdir
int
Sample Command:
Sample Response:
Note: To verify whether directory is created successfully run List all files command.
Sharing a File on dStorage
Zbox supports sharing of files using auth tokens. Sharing can be encrypted or not encrypted for this section we will describe the non-encrypted share functionality. For encrypted shared functionality have a look at Private Sharing .
./zbox share command is used to generate an authtoken that provides authorization to the holder to the specified file on the remotepath. Parameters for sharing the file can be viewed using the ./zbox share --help
The parameters are:
allocation
yes
allocation id
string
clientid
no
id of user to share file with, leave blank for public share
string
encryptionpublickey
no
public key of the client to share file with, required if clientId
string
expiration-seconds
no
seconds before auth ticket expires
int
remotepath
yes
remote path of file to share
string
revoke
no
revoke share for remote path
flag
Command:
The response will yield an Auth token which can be shared publicly to users for downloading the file.
Sample Response:
Note: Use an export command and mark your Auth token to a variable to use with more commands
Downloading the file from dStorage
The ./zbox download can be used to download the file in two ways from dStorage. Using the Allocation ID or Authticket.
Download using Allocation ID and Remotepath
Parameters for downloading the contents of file from dStorage can be viewed using the./zbox download --help
Here are the parameters:
allocation
yes
allocation id
string
authticket
no
auth ticked if not owner of the allocation, use share to get auth ticket
string
blockspermarker
no
download multiple blocks per marker
10
int
commit
no
save metadata to blockchain
false
boolean
endblock
no
download until specified block number
int
localpath
yes
local path to which to download the file to
file path
remotepath
yes
remote path to which the file was uploaded
string
rx_pay
no
authticket must be valid, true = sender pays, false = allocation owner pays
false
boolean
startblock
no
start download from specified block
int
thumbail
no
only download the thumbnail
false
boolean
Command:
Sample Response:
If you are not able to download files and getting Error in file operation: File content didn't match with uploaded file response check Error 2 in Tips and Troubleshooting Section.
Download using Authticket
The necessary flags for downloading the file using the Authticket is auth ticket and local path. Get the AuthToken for the file on dstorage from another user and mark it to a variable. We will use auth token created and marked to AUTH variable for the file we shared in sharing a file on dStorage section .
Command:
Response:
As you can see Zbox has identified the file(info.txt) based on the auth token and didn't require any remote path and allocation id.
Deleting a File from dStorage
Deleting the file on dStorage can be done using the ./zbox delete command. Parameters for deleting the contents of the file on dStorage can be seen by typing ./zbox delete --help
Here are the parameters:
allocation
yes
allocation id
string
remotepath
yes
remote path of file to delete
string
commit
no
save meta data to blockchain
false
boolean
Command:
Sample Response:
Verify whether the updated txt file is available on dstorage using the list command
Response:
As you can see the file is not showing in remote directory which means the file has been successfully deleted.
Copy File to Another Existing Folder in dStorage
Use ./zbox copycommand to copy file to another folder path in dStorage. Only the owner of the allocation can copy an object.
By using ./zbox copy --help .You can have a look at the parameters:
allocation
yes
allocation id
string
commit
no
save metadata to blockchain
false
boolean
remotepath
yes
remote path of object to copy
string
destpath
yes
destination, an existing directory to copy object
string
Required Flags for copying the file to another folder are Allocation ID, the remote path to the existing file on dstorage, and the new destination path/folder path to copy a file to.
Steps :
Verify whether the sync.txt file is available in the remote path using list command.
Response:
2. Use the ./zbox copy command to copy files.
Successful Response:
3. Verify whether the file has copied using:
Response:
Moving Files Between Directories in dStorage
The ./zbox move command can be used to move files between folder paths in dStorage. The parameters required for the move command can be viewed using ./zbox move --help
allocation
yes
allocation id
string
commit
no
save metadata to blockchain
false
boolean
remotepath
yes
remote path of object to copy
string
destpath
yes
destination, an existing directory to copy object
string
The required parameters are Allocation ID , dest path and remote path of the moving file . Steps:
Verify whether the directories are available in the remote path using the list command.
Response:
As you can see there are two directory paths available /myfiles and myfiles1.We will move the info.txt file available remotepath /myfiles to the destpath /myfiles1 .
2. Use the ./zbox move command
Successful Response:
3. Verify whether the file has moved by listing the directory path using:
Response:
Renaming Files in dStorage
./zbox rename command can be used to rename files already existing in dStorage.Only the allocation's owner can rename a file. Parameters for the command can be viewed using ./zbox rename --help
allocation
yes
allocation id
string
commit
no
save metadata to blockchain
false
boolean
destname
yes
new neame of the object
string
remotepath
yes
remote path of object, do not use with authticket
string
Command:
Response:
Verify by listing the remote path using
Response:
List files from Blobbers
./zbox listcommand is used to list all the files from a specified directory on dStorage. Use ./zbox list --help to know the possible parameters.
allocation
yes
allocation id
string
lookuphash
adjust storage expiration time
string
remotepath
yes
Remote Path of directory on dStorage
string
Command :
Sample Response :
Get Stats for file in dStorage
./zbox stats command can be used to get upload, download and challenge information for a file. Parameters for the command can be viewed using ./zbox stats --help
allocation
yes
allocation id
string
json
no
print result in json format
false
boolean
remotepath
yes
file of which to get stats
string
Steps:
Verify whether the file exists at the remote path using
Response:
2. Run the stats command using
Sample Response:
Get metadata for stored file
./zbox meta command is used to get meta data for the given file in dStorage . Use ./zbox meta --help to know more about possible flags.
allocation
yes
allocation id
string
authticket
no
auth ticked if not owner of the allocation
string
json
no
print result in json format
false
boolean
lookuphash
no
hash of object, use with auth ticket
string
remotepath
no
remote path of object, do not use with authticket
string
Command:
Sample Response:
Get Download cost
./zbox get-download-cost determines the cost for downloading the remote file from dStorage. The client must either be the owner, a collaborator or be using an auth ticket.
allocation
yes
allocation id
string
authticket
no
auth ticket to use if not the owner
string
lookuphash
no
hash of remote file, use with auth ticket
string
remotepath
yes
file of which to get stats, use if owner
string
Command:
Sample Response:
Get Upload cost
./zbox get-upload-cost determines the cost for uploading a local file on dStorage. --duration Ignored if --end true, in which case the cost of upload calculated until the allocation expires.
allocation
yes
allocation id
string
duration
no
duration for which to upload file
duration
end
no
upload file until allocation expires
false
boolean
localpath
yes
local of path to calculate upload
file path
Command:
Response:
Get Differences
./zbox get-diff command returns the differences between the local files specified by localpath and the files stored on the root remotepath of the allocation.localcache flag can also be specified to use the local cache of remote snapshot created during Sync for file comparison.
allocation
yes
allocation id
string
excludepath
no
remote folder paths to exclude during syncing
string array
localcache
no
local cache of remote snapshot
string
localpath
yes
local directory to sync
string
Command:
Response :
Formatted Shortened Response :
Sync
./zbox sync command is used to sync all files in the local folder. Creation, deletion and updation will be done automatically. Parameter details can be viewed using the ./zbox sync --help command.
Here are the Parameters:
allocation
yes
allocation id
string
commit
no
commet metadata to blockchain
false
boolean
encryptpath
no
local directory path to be uploaded as encrypted
false
boolean
excludepath
no
paths to exclude from sync
string array
localcache
no
local cache of remote snapshot. Used for comparsion with remote. After sync will be updated.
string
localpath
yes
local directory to which to sync
file path
uploadonly
no
only upload and update files
false
boolean
Example:
Let's sync files stored in the local directory .The local directory for sync in the command below is mapped to local variable. Variable cache is marked to local cache text file located inside the local directory..
Command:
Response:
Verify whether all the files located in the local directory are synced to dStorage using
Batch upload files to dStorage
./zbox sync command with --uploadonly flag can be used to upload all the files in local folder to dStorage. Parameter details can be viewed using the ./zbox sync --help
Here are the Parameters:
allocation
yes
allocation id
string
commit
no
commet metadata to blockchain
false
boolean
encryptpath
no
local directory path to be uploaded as encrypted
false
boolean
excludepath
no
paths to exclude from sync
string array
localcache
no
local cache of remote snapshot. Used for comparsion with remote. After sync will be updated.
string
localpath
yes
local directory to which to sync
file path
uploadonly
no
only upload and update files
false
boolean
Example:
We will create three txt files and move them into a directory to specify it as a local path for the ./zbox sync command.
Here is the list of contents of the directory:
Now by using the ./zbox sync command with --uploadonlyflag we will batch upload these files to the dStorage .Variable local is marked to local directory containing all the files for upload. Variable cache is marked to local cache text file located inside the local directory.
Command:
Response
Verify whether the files are uploaded using
Response:
All the files located in the local directory should be listed in the remotepath.
Private File Sharing
Private sharing of files can be implemented using the ./zbox share command.Parameters can be viewed using./zbox share --helpare shown below:
allocation
yes
allocation id
string
clientid
no
id of user to share file with, leave blank for public share
string
encryptionpublickey
no
public key of the client to share file with, required if clientId
string
expiration-seconds
no
seconds before auth ticket expires
int
remotepath
yes
remote path of file to share
string
revoke
no
revoke share for remote path
flag
Note: Private File Sharing only works with encrypted files.
Let's consider two users (User A and B) and User A wants to privately share the info.txt file located in myfiles/info.txt directory in dStorage with User B.
1.User A will upload the encrypted info.txt file. For instructions on how to upload an encrypted file check Uploading an Encrypted file on dStorage.
2. User B has to share his client id and encryption public key which can be retrieved using thegetwalletcommand. Here we are showing the output of the UserB getwallet command
Command:
Response:
3.Let's export the User B clientid and public encryption key to a variable named UserB and PUBLIC_KEY_UserB
4.Now use the ./zbox share command with above exported variables .Also specify the allocation and remote path for the file shared by User A which is myfiles/info.txt .
So the sample command for the UserA is:
Response:
The Response provides an encrypted auth token string that only UserB can use for private download. Any other user using this AuthToken will not be able to download the file.
5.Let's use ./zbox download using Authtoken provided on UserB computer to privately download the file. Necessary Flags are Authtoken and local path to save the file. Make sure to point encrypted auth token using the export AUTH=<Auth token> before running the download command.
Response:
6.To check sharing details and verify whether the file is encrypted. You can also base decode the Authtoken using:
Response:
The response will provide all the private sharing details and encryption key for the shared file under re_encryption_key field.
Cancel Sharing of File for a Particular User
./zbox share --revoke is used to cancel the share for particular buyer that performed by the seller through Private File Sharing.
Note : ./zbox share --revokeonly works for encrypted files on dStorage.
Required Parameters for Cancel Share are :
allocation
yes
allocation id
string
remotepath
yes
remote path of an encrypted file that was earlier shared with the user
duration
clientid
yes
Wallet Client ID of the user whose share access has to be revoked
string
Command :
Response :
Private Directory Sharing
Private sharing of directories can be implemented using the ./zbox share command.Parameters can be viewed using./zbox share --helpare shown below:
allocation
yes
allocation id
string
clientid
no
id of user to share file with, leave blank for public share
string
encryptionpublickey
no
public key of the client to share file with, required if clientId
string
expiration-seconds
no
seconds before auth ticket expires
int
remotepath
yes
remote path of file to share
string
revoke
no
revoke share for remote path
flag
Note: Private Directory Sharing works for both encrypted and non-encrypted files
Let's consider two users (User A and B) and User A wants to privately share the docs directory in dStorage with User B.
1.User A should check whether the docsdirectory exists on dStorage using the list command.
Command :
Response :
2. Now User B has to share his client id and encryption public key which can be retrieved using thegetwalletcommand. Here we are showing the output of the UserB getwallet command
Command:
Response:
3.Let's export the User B clientid and public encryption key to a variable named UserB and PUBLIC_KEY_UserB
4.Now use the ./zbox share command with above exported variables .Also specify the allocation and remote path for the directory .
Response:
The Response provides an encrypted auth token string that only UserB can use for private download. Any other user using this AuthToken will not be able to download the file.
5. The string is encoded , to know the sharing details of the string UserB can decode it using the command :
Response :
6. Now UserB can download files inside the shared directory with the AUTH token. First point the encrypted auth token to a variable using the export command.
7. Download the files inside the directory by using the following command
Response:
Note : The allocation ID required in the command above can be retrieved from decoding the AUTH string. For remote path the actual path for the shared directory should be specified.
If you are not able to download files and getting Error in file operation: File content didn't match with uploaded file response check Error 2 in Tips and Troubleshooting Section.
Repair a file on dStorage
Use ./zbox start-repair command to repair a file on dStorage. Parameters can be viewed using the --help flag.
allocation
yes
allocation id
string
repairpath
yes
remote path of files on dStorage to repair
string
rootpath
yes
File path for local files. Files from rootpath will be compared with files on dStorage for data blocks error.
string
Sample Command:
Response:
In the above response the total number of files repaired were 0 which means all the data blocks for the files are not altered or damaged.
Rollback
Use ./zbox rollback to rollback to a previous state of allocation. This is helpful when you want to rollback to previous version of files you updated on allocation using Update allocation.
--allocation
Provide Allocation ID for rollback.
string
--h,--help
help for rollback
int
Sample Command:
Sample Response:
Total Stored Data
./zbox total-stored-data returns the summation of all the Size fields in all the files stored across blobbers.
Sample Command:
Sample Response:
Asking the receiver to pay for download (RX_pay)
RX pay is a specified boolean flag in ./zbox download command which asks the file receiver to pay for their file download.
Parameters for --rx_pay flag can be viewed using the zbox download --help.
allocation
yes
allocation id
string
authticket
no
auth ticked if not owner of the allocation, use share to get auth ticket
string
blockspermarker
no
download multiple blocks per marker
10
int
commit
no
save metadata to blockchain
false
boolean
endblock
no
download until specified block number
int
localpath
yes
local path to which to download the file to
file path
remotepath
yes
remote path to which the file was uploaded
string
rx_pay
no
authticket must be valid, true = sender pays, false = allocation owner pays
false
boolean
startblock
no
start download from specified block
int
thumbail
no
only download the thumbnail
false
boolean
As we can see that RX pay requires download by auth ticket and a rx_pay boolean set to true .
Example: Let's take two users (User A and User B). User A is the seller of the file downloaded by User B (receiver).
1.User A will upload a file on dstorage and share an authtoken for the file. For example we are uploading
info1.txt file located on myfiles directory of dStorage which User B will download and pay for himself using --rxpay.
2.To make the file available for pay User A has to set the--attr-who-pays-for-reads for upload command to 3rd_party (by default it is set to "owner")
Sample Command:
3.To verify whether downloads payer are properly configured use:
Response:
In the response, you will notice that the downloads payers section for info1.txt are successfully set to 3rd-party.
4.To know how to share a file on dStorage and get an AuthToken for a file have a look at the sharing a file on dStorage .
5.Once the auth token is created and marked to a variable, share it with the receiver(UserB) who will run the .zbox download command to download the file with --rx_pay flag.
Command for User B:
Response:
To verify whether (UserB) has paid successfully for the download check the wallet and read pool balance for change using ./zwallet getbalance and /zbox rp-info.
Note: Do unlock read tokens using rp-unlock before using ./zwallet getbalanceto let the download money reflect in wallet.
Video Streaming
Video streaming with Zboxcli can be implemented with players for different operating platforms(iOS, Android, Mac). Zbox CLI does not have a player itself and uses the following helping functions :
downloadFileByBlocks - properly returns file-chunks with correct byte range,.
getFileMeta - returns actuaBlockNumbers and actualFileSize (exclude thumbnail size)
getFileMetaByAuth - same updates as getFileMeta
listAllocation - returns actuaBlockNumbers and actualFileSize (exclude thumbnail size)
For all the Platforms streaming implementation using ZboxCLi has the following flow:
When the user starts the video player (ExoPlayer for Android or AVPlayer for iOS), A ZChainDataSource starts chunked download and requests chunks of video from the buffer(a Middleman between streaming player and Zbox).
After the arrival of the first chunk, the player starts requesting more chunks from the buffer which request the Zbox SDK .Zbox SDK which is built using GO makes use of the downloadFileByBlocks method to reliably download large files by chunking them into a sequence of parts that can be downloaded individually. Once the blocks are downloaded they are read into input streams and added to the media source of the streaming player.
The task of downloading files and writing them to buffer using ZboxSDK happens constantly and If player request random bits of video they are delivered instantly by the buffer.
In a case, if the player didn't receive chunks (for example it's still not downloaded), then the player switches to STALE state and the video stream will pause. During the STALE state, player tries to make multiple requests for chunks, if didn't receive a response the video stream stops.
To understand how Zbox CLI provides downloading of files by blocks check the below demonstration:
For instance, Let's consider an allocation which has a audio.mp3file stored on dStorage. Make sure the file has a large size(more than 64 kB(64000 bytes) ) to download the file by blocks. The size and other attributes of the sampleaudio.mp3file can be viewed using
Response:
Here we can see audio.mp3 file has size (5993286) bytes and has 92 blocks. If we want to download a certain number of blocks for the info.txt file we can use --endblock or --startblock flag with ./zbox download command. Other flags for download can be viewed using ./zbox download --help
--allocation
yes
Allocation ID
--authticket
no
Auth ticket for the file to download if you dont own it
-b, --blockspermarker
no
Pass this option to download multiple blocks per marker (by default 10)
--delay
no
Pass segment duration to generate media playlist(m3u8). only works with --live. default duration is 5s. (default 5)
-e,--endblock
yes
Pass this option to download till specific block number
-h, --help
no
help for download
--live
no
Start m3u8 downloader,and automatically generate media playlist(m3u8) on --localpath
--localpath
yes
Local path of file to download
--lookuphash
no
The remote lookuphash of the object retrieved from the list
--remotepath
yes
Remote path to download
-s, --startblock
yes
Pass this option to download from specific block number
-t,--thumbnail
no
Pass this option to download only the thumbnail
To download only 3 blocks of audio.mp3 file we specify --startblock and--endblock with an integer value of 1 and 3. --blockspermarkerflag can also be specified to download multiple blocks at a time(default is 10).
The whole download command for downloading till 3rd block of audio.mp3 file would be:
Response:
As we can see, the downloaded file size(393216) is quite less than the original(2996198) which means zbox has downloaded some blocks of the file.
Feed
Use feed command to automatically download segment files from remote live feed with --downloader-args "-q -f best"
encode them into new segment files with
--delayand--ffmpeg-args, and upload.please use
youtube-dl -F https://www.youtube.com/watch?v=pC5mGB5enkwto list formats of video (see below).
--downloader-args "-f 22" dowloads video with 22 mp4 1280x676 720p 2117k , avc1.64001F, 30fps, mp4a.40.2 (44100Hz) (best)
The user must be the owner of the allocation.You can request the file be encrypted before upload, and can send thumbnails with the file.
allocation
yes
allocation id, sender must be allocation owner
string
encrypt
no
encrypt file before upload
false
boolean
localpath
yes
local path of segment files to download, generate and upload
file path
remotepath
yes
remote path to upload file to, use to access file later
string
thumbnailpath
no
local path of thumbnaSil
file path
chunknumber
no
how many chunks should be uploaded in a http request
1
int
delay
no
set segment duration to seconds.
5
int
feed
no
set remote live feed to url.
false
url
downloader-args
no
pass args to youtube-dl to download video. default is "-q -f best".
-q -f best
Sample Command:
Last updated