Code

S3 Search Improvements#1391

Code

Merged

ssalinas merged 26 commits intomasterHubSpot/Singularity:masterfrom s3_reworkHubSpot/Singularity:s3_reworkCopy head branch name to clipboardJan 31, 2017

+1,604-551

Commits

Commits on Dec 14, 2016

• allows setting storage class at upload time

  wsorenson

  committed
  57f3ffe

• allow each additional file to specify s3 storage class

  wsorenson

  committed
  e26e8dc

• add debug stmt

  wsorenson

  committed
  391295c

Commits on Jan 6, 2017

• fix merge conflicts

  ssalinas

  committed
  bb55ae8

• Move S3 file configuration to SingularityService

  ssalinas

  committed
  13d2fe9

Commits on Jan 9, 2017

• jets3t -> aws sdk

  ssalinas

  committed
  84e112b

• use int here

  ssalinas

  committed
  fcde16e

• Merge pull request #1394 from HubSpot/jets_to_aws

  Show description for 9db459b
  ssalinas

  authored
  9db459b

Commits on Jan 10, 2017

• cleanup needs to know defaults for transition to new format

  ssalinas

  committed
  0c9659a

• cleaner check for default service log name

  ssalinas

  committed
  508ee73

• more working on getting executor cleanup to run proerply

  ssalinas

  committed
  6811cb0

• this should be in cleanup configuration now s3

  ssalinas

  committed
  3e0d3cf

Commits on Jan 11, 2017

• add support for extra buckets to search for logs

  ssalinas

  committed
  ace58e7

• Implement pagination for S3 endpoint using continuation tokens

  Show description for 74e07a9
  ssalinas

  committed
  74e07a9

• Merge pull request #1395 from HubSpot/s3_pagination

  Show description for 8242609
  ssalinas

  authored
  8242609

• Merge branch 's3_rework' of github.com:HubSpot/Singularity into s3_rework

  ssalinas

  committed
  8d52018

• search the additional buckets when present

  ssalinas

  committed
  8861141

• do not search prefixes that will return duplicate results

  ssalinas

  committed
  c04a98e

Commits on Jan 17, 2017

• don't need group here after all

  ssalinas

  committed
  95d40fd

• fix prefix trim

  ssalinas

  committed
  ddb6e43

• need to use length + startsWith, not contains

  ssalinas

  committed
  9de457d

Commits on Jan 18, 2017

• multipart upload needs key as well

  ssalinas

  committed
  8d157e3

• Also add file arg

  ssalinas

  committed
  6fb4c2c

Commits on Jan 20, 2017

• Set metadata here as well

  ssalinas

  committed
  5b23fe9

Commits on Jan 23, 2017

• user metadata is separate

  ssalinas

  committed
  d426dea

Commits on Jan 31, 2017

• add docs for s3 changes

  ssalinas

  committed
  5e6d768