# 使用S3Cmd

由於s3cmd的command眾多,以下我們僅介紹幾個常用的命令,如需詳細命令列表,可執行s3cmd以便列出完整的help說明。

透過s3cmd訪問RADOSGW時,由於我們並未配置客戶端憑證,故下達s3cmd時必須包含option: --no-check-certificate否則會因為未配置憑證而報錯。

例如:

$ s3cmd --no-check-certificate mb s3://mybucket

若不想每次使用s3都帶此option,可以透過建指令別名:alias來簡化輸入。

$ alias s3cmd='s3cmd --no-check-certificate'

讓此別名永久有效

$ echo "alias s3cmd='s3cmd --no-check-certificate '" >> ~/.bashrc

$ source ~/.bashrc

# 1. 建立bucket

$ s3cmd mb s3://mybucket

Bucket 's3://mybucket/' created

# 2. 上傳檔案至bucket中

$ s3cmd put 1_hello_tensorflow.ipynb s3://mybucket

upload: '1_hello_tensorflow.ipynb' -> 's3://mybucket/1_hello_tensorflow.ipynb' [1 of 1]

25023 of 25023 100% in 0s 207.76 kB/s done

# 3. 列出某一路徑下的檔案或bucket

$ s3cmd ls s3://mybucket

2018-11-15 03:40 25023 s3://mybucket/1_hello_tensorflow.ipynb

# 4. 從bucket中取得檔案

$ s3cmd get s3://mybucket/1_hello_tensorflow.ipynb /tmp

download: 's3://mybucket/1_hello_tensorflow.ipynb' ->

'/tmp/1_hello_tensorflow.ipynb' [1 of 1] 25023 of 25023 100% in 0s 108.27 kB/s done

# 5. 刪除bucket中的檔案

$ s3cmd del s3://mybucket/1_hello_tensorflow.ipynb

delete: 's3://mybucket/1_hello_tensorflow.ipynb'

# 6. 刪除bucket

$ s3cmd rb s3://mybucket

Bucket 's3://mybucket/' removed

# 7. Sync檔案

例如: 從s3://testbucket2 sync至/tmp/mybucket目錄

$ s3cmd sync --no-preserve s3://testbucket2 /tmp/mybucket

download: 's3://testbucket2/1_hello_tensorflow.ipynb' -> '/tmp/mybucket/1_hello_tensorflow.ipynb' [1 of 6]

25023 of 25023 100% in 0s 247.52 kB/s done

download: 's3://testbucket2/2_getting_started.ipynb' -> '/tmp/mybucket/2_getting_started.ipynb' [2 of 6]

164559 of 164559 100% in 0s 564.76 kB/s done

download: 's3://testbucket2/3_mnist_from_scratch.ipynb' -> '/tmp/mybucket/3_mnist_from_scratch.ipynb' [3 of 6]

209951 of 209951 100% in 0s 924.68 kB/s done

download: 's3://testbucket2/4_keras_tensorboard_basic.ipynb' ->

'/tmp/mybucket/4_keras_tensorboard_basic.ipynb' [4 of 6]

3348 of 3348 100% in 0s 31.83 kB/s done

download: 's3://testbucket2/eula.3082.txt' -> '/tmp/mybucket/eula.3082.txt' [5 of 6]

17734 of 17734 100% in 0s 55.82 kB/s done

Done. Downloaded 420615 bytes in 1.0 seconds, 401.72 kB/s

$ ls -l /tmp/mybucket

備註: 因S3 object已與文件系統”屬性”一起保存。當您將它們同步到本地目錄時,默認情況下s3cmd sync將嘗試恢復這些屬性,包括uid、gid所有權等。

要防止s3cmd執行此操作,請加上option: --no-preserve,否則您將會看到警告訊息: “WARNING: xxxx not writable: Operation not permitted”

同理,我們亦可從local sync回RADOSGW的bucket。

$ s3cmd sync /tmp/mybucket s3://testbucket2
Last Updated: 9/22/2022, 7:29:43 PM