AI, startup hacks, and engineering miracles from your friends at Faraday

Finding public S3 objects

Seamus Abshere on

There's lot of concern over public objects in S3 buckets. Now Amazon gives you a way to lock down entire buckets - but what if you legitimately have a mix of public and private objects?

Ruby script to find public S3 objects

First get 2 gems:

source 'https://rubygems.org' do
  gem 'aws-sdk'
  gem 'thread'
end

Then the script itself:

# find_public_s3_objects.rb
require 'aws-sdk-s3'
require 'thread/pool'
BUCKET = ARGV[0] or raise("expected bucket")
s3 = Aws::S3::Resource.new(region: 'us-east-1')
count = 0
pool = Thread.pool 8
mutex = Mutex.new
s3.bucket(BUCKET).objects.each do |object|
  pool.process do
    grants = object.acl.grants
    mutex.synchronize do
      count += 1
      if count % 100 == 0
        $stderr.write "#{count}..."
      end
    end
    if grants.map { |x| x.grantee.uri }.any? { |x| x =~ /AllUsers/ }
      mutex.synchronize do
        puts object.key
      end
    end
  end
end
pool.shutdown

Then you run it like this:

bundle exec ruby find_public_s3_objects.rb my-bucket-name

It's much faster than the bash-based example on StackOverflow.