Skip to main content

How to mount the S3 bucket on to the Mac

How to mount the S3 bucket on to the Mac

Description

S3 is very powerful architecture. But it cannot use like normal directory. However, we have the solution to mount S3 on Mac PC. It's to use Storage Gateway.

Archtecture Image

ー ー ー

Premise

Storage Gateway: Requirements for Amazon EC2 instance types https://docs.aws.amazon.com/storagegateway/latest/userguide/Requirements.html#requirements-hardware-ec2

[Recommended for file gateway types]
・General-purpose instance family – m4 or m5 instance type.
・Compute-optimized instance family – c4 or c5 instance types. Choose the 2xlarge instance size or higher to meet the required RAM requirements.
・Memory-optimized instance family – r3 instance types.
・Storage-optimized instance family – i3 instance types.

[Recommended for cached volumes and tape gateway types]
・General-purpose instance family – m4 or m5 instance types. We don't recommend using the m4.16xlarge instance type.
・Compute-optimized instance family – c4 or c5 instance types. Choose the 2xlarge instance size or higher to meet the required RAM requirements.
・Storage-optimized instance family – d2, i2, or i3 instance types.

Storage Gateway: Storage requirements https://docs.aws.amazon.com/storagegateway/latest/userguide/Requirements.html#requirements-storage

In addition to 80 GiB disk space for the VM, you also need additional disks for your gateway.

The following table recommends sizes for local disk storage for your deployed gateway.

Gateway Type	Cache (Minimum)	Cache (Maximum)	Upload Buffer (Minimum)	Upload Buffer (Maximum)	Other Required Local Disks
File gateway	150 GiB	64 TiB	—	—	—
Gateway Type Cache (Minimum) Cache (Maximum) Upload Buffer (Minimum) Upload Buffer (Maximum) Other Required Local Disks
File gateway 150 GiB 64 TiB - - -

You'll check any other requirements such as network in the public document.

Getting Started

  1. Setup the Storage Gateway

    1. Open the Storage Gateway console

    2. Click on the Get Started

    3. Click on the Next after selected the File gateway

    4. Click on the Launch instance and setting EC2 instance that it satisfied the requirements

      NOTE: Storage Gateway is using dedicated AMI. So, I recommend you to create EC2 instance from this link.

    5. Click on the Next after selected the Amazon EC2

    6. Click on the Next after selected the Public

    7. Click on the Connect to gateway after input EC2 IP address

    8. Click on the Activate gateway after input gateway name

    9. Create EBS volume and attach to EC2 instance for Storage Gateway

    10. Click on the Configure logging

    11. Click on the Save and continue

  2. Create the file sharing
    1. Create the S3 bucket for share with Storage Gateway
    2. Open the Storage Gateway console
    3. Click on the Create file share
    4. Click on the Create file share after input S3 bucket name and select the NFS
    5. Click on the Next
    6. Click on the Create file share
  3. Mount on the Mac
    1. Create the directory for mounting
    2. Mount to the directory by using the following command
      mount_nfs -o vers=3,nolock,hard -v [Gateway public IP address]:/[S3 bucket name] [directory path]

Finally

This solution has following these costs.

  • EC2
  • EBS
  • S3
  • data communication and requests (including Storage Gateway)

So, you need to consider that you tolerate the cost of this solution or not.

Label:

Popular posts from this blog

How to calculate the infinite product by SQL

How to calculate the infinite product by SQL Description I needed to calculate the infinite product in the creating ETL script. So I thought the solution to calculate it in the SQL. The following way is the solution I thought. Premise Use the standard SQL Code WITH sample_table AS ( SELECT 2 AS n1, 4 AS n2 UNION ALL SELECT 4 , 8 UNION ALL SELECT 10 , 20 UNION ALL SELECT 10 , 20 ) SELECT * FROM sample_table UNION ALL SELECT CAST ( EXP ( SUM ( LOG (n1))) AS INT64) AS inf_product1, CAST ( EXP ( SUM ( LOG (n2))) AS INT64) AS inf_product2 FROM sample_table The infinite product (∏) is able to convert to the total addition formula. ∏n1 ≒ e Σlog e n1 ∏n2 ≒ e Σlog e n2 Finally Actually, this calculated value is an approximation. So if you want an accurate number, this solution cannot use.
Read more

Create multiple sub commands with [Click]

Create multiple sub commands with [Click] Description I'd like to create the script that includes some options such as the awscli . We can create it easy to use the Click library. So, I write a sample code about creating sub-command with Click . Premise Python : >= 3.7 Click : >= 7.0 Code import click def main(): cmd() @click.group() def cmd(): """First layer sub-command group """ pass @cmd.group() def sub_command_1(): """Second layer sub-command group """ pass @cmd.group() def sub_command_2(): """Second layer sub-command group """ pass @sub_command_1.command() def sub_command_1_1(): print("Hello, subcommand 1 under the 1st layer subcommand 1") @sub_command_1.command() def sub_command_1_2(): print("Hello, sub...
Read more