can't access govcloud s3 buckets

[david sharpe]

s3 connections to normal aws buckets work correctly. i cannot access govcloud s3 buckets. suspect there is a different s3 endpoint to use or another setting.

recommend offer both S3 and S3 Govcloud as separate options for Protocol and make it automatically configure the correct endpoint.

always failing to validate username or password even though IAM user credentials work correctly on the server to access the bucket.

[Tim Kosse]

While we do not have access to the GovCloud and have not been able to test it, you should still be able to use FileZilla Pro to access your GovCloud resources.

In the settings dialog of FileZilla Pro on the S3 Providers page, please add the following region to the Amazon S3 provider:

Name: us-west-gov1
Description: AWS GovCloud (US-West)
Endpoints: s3.dualstack.us-gov-west-1.amazonaws.com

To connect, also use s3.dualstack.us-gov-west-1.amazonaws.com as hostname in the Site Manager.

Reference: ​https://docs.aws.amazon.com/govcloud-us/latest/ug-west/using-govcloud-endpoints.html

Please let us know if this works for you.

[david sharpe]

almost. great suggestion. i tried exactly that.

however that setting didn't seem to play into the actual connection string which reverted back to a us-east-1 connection

Status: Retrieving directory listing...
Status: Resolving address of s3.dualstack.us-east-1.amazonaws.com
Status: Connecting to [2600:1fa0:8068:a1c9:34d8:6ddd::]:443...
Status: Connection established, initializing TLS...
Status: Verifying certificate...
Status: TLS connection established, sending HTTP request
Command: GET / HTTP/1.1
Command: Authorization: *
Command: Connection: keep-alive
Command: Host: s3.dualstack.us-east-1.amazonaws.com:443
Command: Keep-Alive: 300
Command: User-Agent: FileZilla/3.40.0
Command: x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command: x-amz-date: 20190205T210105Z
Response: HTTP/1.1 403 Forbidden
Response: x-amz-request-id: 728986D85EEDB2DF
Response: x-amz-id-2: /iIFsgYJqWrSur15fgR3Cm87UWfnIYc56OxXJVNCkMwRqONHom+L81/zQneYIVtuoLKtcEuY2gY=
Response: Content-Type: application/xml
Response: Transfer-Encoding: chunked
Response: Date: Tue, 05 Feb 2019 21:01:05 GMT
Response: Server: AmazonS3
Error: Please verify the user name and password used to connect.
Error: Failed to retrieve directory listing

[Tim Kosse]

At this point a verbose log is needed.

Please start FileZilla fresh, then set the debug log level to 3 on the Debug page in the settings. Last but not least, connect to S3 using the Site Manager and post the resulting contents of the message log.

[david sharpe]

here is the verbose log. i see the correct address is on line 1 then it is overwritten by the time the request is sent out. this looks like it should narrow it down for you.

Trace: CS3ControlSocket::Connect(s3.dualstack.us-gov-west-1.amazonaws.com)
Trace: CControlSocket::SendNextCommand()
Trace: CHttpConnectOpData::Send() in state 0
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpConnectOpData::Reset(0) in state 0
Trace: CS3ControlSocket::List()
Status: Retrieving directory listing...
Trace: CControlSocket::SendNextCommand()
Trace: CS3ListOp::Send() in state 0
Trace: CS3ControlSocket::DoRequest
Trace: S3RequestOp::Send() in state 0
Trace: Requesting ​https://s3.dualstack.us-east-1.amazonaws.com:443/
Trace: CHttpControlSocket::Request()
Trace: CHttpRequestOpData::Send() in state 17
Trace: CHttpRequestOpData::Send() in state 18
Trace: CHttpControlSocket::InternalConnect()
Trace: CHttpControlSocket::ResetSocket()
Trace: CHttpInternalConnectOpData::Send() in state 0
Status: Resolving address of s3.dualstack.us-east-1.amazonaws.com
Status: Connecting to [2600:1fa0:8068:9f89:34d8:6cad::]:443...
Status: Connection established, initializing TLS...
Trace: CTlsSocketImpl::Handshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: TLS Handshake successful
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: AES-128-GCM, MAC: AEAD
Status: Verifying certificate...
Status: TLS connection established, sending HTTP request
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpInternalConnectOpData::Reset(0) in state 0
Trace: CHttpRequestOpData::SubcommandResult(0) in state 18
Trace: CControlSocket::SendNextCommand()
Trace: CHttpRequestOpData::Send() in state 20
Command: GET / HTTP/1.1
Command: Authorization: *
Command: Connection: keep-alive
Command: Host: s3.dualstack.us-east-1.amazonaws.com:443
Command: Keep-Alive: 300
Command: User-Agent: FileZilla/3.40.0
Command: x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command: x-amz-date: 20190206T190921Z
Trace: Finished sending request header. Request has no body
Trace: CHttpRequestOpData::Send() in state 16
Trace: CHttpRequestOpData::ParseHeader()
Response: HTTP/1.1 403 Forbidden
Response: x-amz-request-id: 870A370FA6427FC4
Response: x-amz-id-2: yKizgwCne9C4wmz+mVrMh/dJaQctoDCFQLNVf/tAXz6srnw4iMeeqgI2l1/VznA7npHlpaApLkA=
Response: Content-Type: application/xml
Response: Transfer-Encoding: chunked
Response: Date: Wed, 06 Feb 2019 19:09:21 GMT
Response: Server: AmazonS3
Trace: CHttpRequestOpData::ParseHeader()
Trace: S3RequestOp::OnHeader with response code 403
Trace: Finished a response
Trace: Done reading last response
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpRequestOpData::Reset(0) in state 0
Trace: S3RequestOp::SubcommandResult(0) in state 2
Trace: CControlSocket::ResetOperation(2)
Trace: S3RequestOp::Reset(2) in state 2
Trace: CS3ListOp::SubcommandResult(2) in state 0
Error: Please verify the user name and password used to connect.
Trace: CControlSocket::ResetOperation(1026)
Trace: CS3ListOp::Reset(1026) in state 0
Error: Failed to retrieve directory listing
Trace: Idle socket got closed
Trace: CHttpControlSocket::ResetSocket()

[Tim Kosse]

I think the region name mention in my first reply might not be correct. Could you please try entering us-west-gov-1 (note the dash between gov and 1) as region name in the settings dialog?

[Tim Kosse]

In case used copy&paste to enter the configuration, make sure there's no leading/trailing whitespace in any of the fields.

[david sharpe]

you mean like this?

Trace: CS3ControlSocket::Connect(us-west-gov-1)
Trace: CControlSocket::SendNextCommand()
Trace: CHttpConnectOpData::Send() in state 0
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpConnectOpData::Reset(0) in state 0
Trace: CS3ControlSocket::List()
Status: Retrieving directory listing...
Trace: CControlSocket::SendNextCommand()
Trace: CS3ListOp::Send() in state 0
Trace: CS3ControlSocket::DoRequest
Trace: S3RequestOp::Send() in state 0
Trace: Requesting ​https://s3.dualstack.us-east-1.amazonaws.com:443/
Trace: CHttpControlSocket::Request()
Trace: CHttpRequestOpData::Send() in state 17
Trace: CHttpRequestOpData::Send() in state 18
Trace: CHttpControlSocket::InternalConnect()
Trace: CHttpControlSocket::ResetSocket()
Trace: CHttpInternalConnectOpData::Send() in state 0
Status: Resolving address of s3.dualstack.us-east-1.amazonaws.com
Status: Connecting to [2600:1fa0:8050:1d89:34d9:10e::]:443...
Status: Connection established, initializing TLS...
Trace: CTlsSocketImpl::Handshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: TLS Handshake successful
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: AES-128-GCM, MAC: AEAD
Status: Verifying certificate...
Status: TLS connection established, sending HTTP request
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpInternalConnectOpData::Reset(0) in state 0
Trace: CHttpRequestOpData::SubcommandResult(0) in state 18
Trace: CControlSocket::SendNextCommand()
Trace: CHttpRequestOpData::Send() in state 20
Command: GET / HTTP/1.1
Command: Authorization: *
Command: Connection: keep-alive
Command: Host: s3.dualstack.us-east-1.amazonaws.com:443
Command: Keep-Alive: 300
Command: User-Agent: FileZilla/3.40.0
Command: x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command: x-amz-date: 20190206T195433Z
Trace: Finished sending request header. Request has no body
Trace: CHttpRequestOpData::Send() in state 16
Trace: CHttpRequestOpData::ParseHeader()
Response: HTTP/1.1 403 Forbidden
Response: x-amz-request-id: 0B5248EBD7E968E8
Response: x-amz-id-2: peSKUy09tpgg2nOcFlJTopPnujGQ4XWxOM3MloIiLe1R/of1o11O84/BrPIclbjuZGH+VE281eU=
Response: Content-Type: application/xml
Response: Transfer-Encoding: chunked
Response: Date: Wed, 06 Feb 2019 19:54:34 GMT
Response: Server: AmazonS3
Trace: CHttpRequestOpData::ParseHeader()
Trace: S3RequestOp::OnHeader with response code 403
Trace: Finished a response
Trace: Done reading last response
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpRequestOpData::Reset(0) in state 0
Trace: S3RequestOp::SubcommandResult(0) in state 2
Trace: CControlSocket::ResetOperation(2)
Trace: S3RequestOp::Reset(2) in state 2
Trace: CS3ListOp::SubcommandResult(2) in state 0
Error: Please verify the user name and password used to connect.
Trace: CControlSocket::ResetOperation(1026)
Trace: CS3ListOp::Reset(1026) in state 0
Error: Failed to retrieve directory listing

[david sharpe]

oh i got it this time - still trying to connect to east

Trace: CS3ControlSocket::Connect(s3.dualstack.us-west-gov-1.amazonaws.com)
Trace: CControlSocket::SendNextCommand()
Trace: CHttpConnectOpData::Send() in state 0
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpConnectOpData::Reset(0) in state 0
Trace: CS3ControlSocket::List()
Status: Retrieving directory listing...
Trace: CControlSocket::SendNextCommand()
Trace: CS3ListOp::Send() in state 0
Trace: CS3ControlSocket::DoRequest
Trace: S3RequestOp::Send() in state 0
Trace: Requesting ​https://s3.dualstack.us-east-1.amazonaws.com:443/
Trace: CHttpControlSocket::Request()
Trace: CHttpRequestOpData::Send() in state 17
Trace: CHttpRequestOpData::Send() in state 18
Trace: CHttpControlSocket::InternalConnect()
Trace: CHttpControlSocket::ResetSocket()
Trace: CHttpInternalConnectOpData::Send() in state 0
Status: Resolving address of s3.dualstack.us-east-1.amazonaws.com
Status: Connecting to [2600:1fa0:80c0:1290:34d8:a115::]:443...
Status: Connection established, initializing TLS...
Trace: CTlsSocketImpl::Handshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: CTlsSocketImpl::ContinueHandshake()
Trace: TLS Handshake successful
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: AES-128-GCM, MAC: AEAD
Status: Verifying certificate...
Status: TLS connection established, sending HTTP request
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpInternalConnectOpData::Reset(0) in state 0
Trace: CHttpRequestOpData::SubcommandResult(0) in state 18
Trace: CControlSocket::SendNextCommand()
Trace: CHttpRequestOpData::Send() in state 20
Command: GET / HTTP/1.1
Command: Authorization: *
Command: Connection: keep-alive
Command: Host: s3.dualstack.us-east-1.amazonaws.com:443
Command: Keep-Alive: 300
Command: User-Agent: FileZilla/3.40.0
Command: x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Command: x-amz-date: 20190206T195541Z
Trace: Finished sending request header. Request has no body
Trace: CHttpRequestOpData::Send() in state 16
Trace: CHttpRequestOpData::ParseHeader()
Response: HTTP/1.1 403 Forbidden
Response: x-amz-request-id: D6F676E0299C9ED1
Response: x-amz-id-2: 1iKzrrVO6njx3HgVnhYGaCVDXGU7eblA9jh7MZYLiMwHJCRxJn+zqlhqdiL0yNBkRPi7BfKvMGg=
Response: Content-Type: application/xml
Response: Transfer-Encoding: chunked
Response: Date: Wed, 06 Feb 2019 19:55:40 GMT
Response: Server: AmazonS3
Trace: CHttpRequestOpData::ParseHeader()
Trace: S3RequestOp::OnHeader with response code 403
Trace: Finished a response
Trace: Done reading last response
Trace: CControlSocket::ResetOperation(0)
Trace: CHttpRequestOpData::Reset(0) in state 0
Trace: S3RequestOp::SubcommandResult(0) in state 2
Trace: CControlSocket::ResetOperation(2)
Trace: S3RequestOp::Reset(2) in state 2
Trace: CS3ListOp::SubcommandResult(2) in state 0
Error: Please verify the user name and password used to connect.
Trace: CControlSocket::ResetOperation(1026)
Trace: CS3ListOp::Reset(1026) in state 0
Error: Failed to retrieve directory listing
Trace: Idle socket got closed
Trace: CHttpControlSocket::ResetSocket()

[david sharpe]

have tried several other variations. thinking you just have code that forces it to use your east endpoint

[Tim Kosse]

Could you please post a screenshot of the S3 Providers page in the settings dialog, showing the added row for the us-west-gov-1 region?

[david sharpe]

site manager screenshot

[david sharpe]

attached.

[Tim Kosse]

I see, that's not the settings dialog. In the main menu, go to Edit -> Settings, inside go to the S3 Providers page.

[david sharpe]

settings - s3

[david sharpe]

attached - i see more under these - are you thinking maybe add gov to this list?

[Tim Kosse]

Yes, as mentioned earlier:

please add the following region to the Amazon S3 provider:

Name: us-west-gov-1
Description: AWS GovCloud (US-West)
Endpoints: s3.dualstack.us-gov-west-1.amazonaws.com

[david sharpe]

ok got your full configuration working. it was close

Response: The authorization header is malformed; the region 'us-west-gov-1' is wrong; expecting 'us-gov-west-1'

Final working answer needs region name to match url

Name: us-gov-west-1
Description: AWS GovCloud (US-West)
Endpoints: s3.dualstack.us-gov-west-1.amazonaws.com

Host Name s3.dualstack.us-gov-west-1.amazonaws.com:

thank you for your help.

can you publish this as a default region? or will i have to keep adding it during updates or?