Brickshelf is shutting down. Let's try to save it.

[tsi]

On 1/31/2025 at 2:02 AM, JeffBuilds said:

Anything you would recommend for PC Users? 

The linked skript on GitHub is working on PCs too, you don't need a MAC - just Python.
I downloaded some brickshelf galleries with it succesfully.

[aFrInaTi0n]

Little update from my side:

I finished with mirroring all images in their original structure of "user/folder/to/image.png".

Some Stats:

Grabbed files: 4682606 / Total Size: 2030 GB

("there are ~300.000 files diff from the homepages counter of public files" -> need to doublecheck this)

 

Next steps:

1. [Finished] Mirroring all images from public hosted brickshelf files
2. [Started] Creating thumbs of all pictures (est time: 1 day)
3. Creating zips on userlevel (est. time: 2-3 days)
4. Host a simple browsable brickshelf-mirror for people with no IT skills at all being able to browse through the directories & pull their user folders with a 1-click (not same scripts / looks , but same content structure)
   (Hint: I am doing this as a private entitity on my private server, from the law perspective it may not be 100% clean. But I think I will take the risk as all pictures were uploaded originally with the intention on sharing them in the internet by the owners - as long as I am not earning money with the hosting and not claiming one of the pictures would be mine I hope nobody would be that mad at me - anyways if I bring up such mirror I will put a contact mail address in for legal claims.)

Other question I have:

How important are the folder descriptions and keywords (and view counts of files) considered? Did not consider those yet.. 

[Toastie]

WOW!!! Incredible @aFrInaTi0n!!!

My goodness.

OK, since you asked: The "date folder created/modified" is of the only additional interest to me - it tells a lot at puts things readily into perspective ... but maybe this is already stored in some data structure?

Once again: Thank you so much for doing this!!! Two TByte of LEGO history ... 

All the best,
Thorsten

[aFrInaTi0n]

I will need to write me a second little scraper to get me the details of the folders.. Are pictures also able to have a description? I think I haven't seen that so far..

 

[Trekkie99]

On 1/31/2025 at 4:34 PM, EWay said:

I'm very hopeful that nothing will be lost, as some here are downloading all the content, others have contacted the estate about buying the site and it sounds like they're responsive... Also per this Reddit post, it sounds like it's all being added to the wayback machine.

 

Very nice to see!!!

[anyUser]

1 hour ago, aFrInaTi0n said:

Are pictures also able to have a description?

As far as I have seen: no. A folder may have one or more keywords (from a list as well as user input) and also a description.

[aFrInaTi0n]

thanks for verfication. @anyUser

[SNIPE]

I think we should shut brickshelf down

But migrate the MOCs to rebrickable

Because brickshelf is absolutey outdated and horrible to use.

Too bad we didn't get to do that before MOCpages went down

in general, people are stupid

[icm]

@aFrInaTi0n: Thanks so much for doing that! I can't wait until you've got all the zips packaged up for user photos, there are quite a few user photo libraries I want to preserve.

[SpacePolice89]

1 hour ago, SNIPE said:

Because brickshelf is absolutey outdated and horrible to use.

I think it's very easy to use Brickshelf, much more so than modern websites.

[aFrInaTi0n]

lil update inbetween, category "don't ever underestimate 20 years of internet history": after being roughly 1/3 through with thumbnail-generation, it is already 89 GB for 300x300 thumbnails... *sigh :D

"compute little server, compute!" :D

[Murdoch17]

3 hours ago, SNIPE said:

I think we should shut brickshelf down

But migrate the MOCs to rebrickable

Because brickshelf is absolutey outdated and horrible to use.

Too bad we didn't get to do that before MOCpages went down

in general, people are stupid

Without getting too far off topic, with MOCpages, there was no warning. MOCpages creator Sean Kenny just decided to pull the plug one day, after years of hacking issues and not responding to requests / offers to help. (Don't get me started on that!) This is a 100% totally different situation and I've got to commend the estate of Kevin M. Loch for trying to keep it running by finding another owner and letting us know what's going on.

Edited February 2 by Murdoch17

[Toastie]

4 hours ago, SNIPE said:

I think we should shut brickshelf down

But migrate the MOCs to rebrickable

Because brickshelf is absolutey outdated and horrible to use.

Too bad we didn't get to do that before MOCpages went down

in general, people are stupid

I am confused ... who is "we"? "We" are migrating the MOCs? I have not only no clue, but I am speechless. And then dump it on rebrickable? "We"???

When a website works for 20 years, it may get outdated - but hey, it still works. Run by one guy - who passed away. Make it 5 people. It works. You upload pictures, they show up and can be referenced. Horrible to use??? - it is for free! What are you asking for? Super service, top-notch 2030+ software? For free?

Well, I agree on the stupidity, I am certainly part of it - which naturally includes you, as it is, as you said, a general thing.

[Brickshelf Archive]

Hi people. Over the past few years I have crawled Brickshelf because I was worried that something like this would happen. I have uploaded the data to S3 and it is currently being served here: https://brickshelf-archive-7c8bea29901f.herokuapp.com/

The data in total comes out to about 2 TB. 

Please note the following:

• Search is primitive and only supports searching by folder ID or member name. Search is case sensitive.
• I'm willing to host the site for a while, as it isn't too expensive, but may ask for donations. If it isn't needed because Brickshelf stays around, I'll take it down.
• If I have time, I'll add some better browsing / searching functionality.

Edited February 3 by Brickshelf Archive

[aFrInaTi0n]

Thanks for your efforts!

 

Your search is not running well - ids are working, but usernames did not work for me, tried with 1-2..

 

Btw: possibly better already think of you really being commited / willing to take the costs privatly or not better think twice about stopping directly - at least it does not sound very promising for one asking for "donations" one sentence later.. dont get me wrong please - just want to say the costs may easily explode if such is offered to the internet - hope you at least set yourself some cost limits in AWS!

[MAB]

21 hours ago, aFrInaTi0n said:

Some Stats:

Grabbed files: 4682606 / Total Size: 2030 GB

I wonder what billing they have on their account. If it is based on bandwidth and if multiple people are doing this, then they could find they get a huge bill this month.

[BrickWild]

Our right to store images & files without having to burn a single dollar bill or two has been one of our most valuable freedoms as a lifestyle.
And now that freedom has been taken away from us.
Remember when Imageshack and Photobucket tried to stop people from posting or sharing porn in the 2010s?
I still post to Flickr but I have to pay for PRO membership every two years. And I have to keep my membership active or risk losing my older images.
Brickshelf will hold a special place in internet history and a place in our hearts. It's truly the end of an ambitious era. 

[MAB]

4 minutes ago, BrickWild said:

Our right to store images & files without having to burn a single dollar bill or two has been one of our most valuable freedoms as a lifestyle.
And now that freedom has been taken away from us.

Someone was paying for that, often through advertising on their own site. But I totally understand companies blocking sharing of image files by links. The company pays for hosting and bandwidth costs but the people viewing the files never see the adverts on the hosting company's site and in many cases would not even know which company is paying to host the images they are viewing. 

[BrickWild]

23 minutes ago, MAB said:

Someone was paying for that, often through advertising on their own site. But I totally understand companies blocking sharing of image files by links. The company pays for hosting and bandwidth costs but the people viewing the files never see the adverts on the hosting company's site and in many cases would not even know which company is paying to host the images they are viewing. 

We understand that Flickr, ImageShack, PhotoBucket and other such similar sites are meant for Hosting, not Long-Term Storage.

As brilliant as that sounds, one of the drawbacks of having resources held hostage is that it ultimately isn't ours anymore.
Back in 2021, Flickr suddenly deleted my account without any statement but I was very lucky to get it back after briefly explaining why their claim was false.

[aFrInaTi0n]

49 minutes ago, MAB said:

I wonder what billing they have on their account. If it is based on bandwidth and if multiple people are doing this, then they could find they get a huge bill this month.

It is up to any provider to take precautions before putting up a service.

But yes you may be right if they have to pay traffic.

On the other hand: if we have more mirroring sites up, it may relieve traffic on them again for the time being..

[MAB]

5 hours ago, aFrInaTi0n said:

It is up to any provider to take precautions before putting up a service.

But yes you may be right if they have to pay traffic.

On the other hand: if we have more mirroring sites up, it may relieve traffic on them again for the time being..

I've seen it discussed on a number of LEGO fan websites. Lots of people seem to be wanting to save brickshelf or at least save the content of brickshelf. If everyone goes on to download the entire archive so they can host elsewhere or keep their own archive without realising others have already done it, then it could cost the family / estate a substantial sum depending on the hosting agreement / billing that he had. I imagine the traffic this month will be huge compared to in the past. It might even cause too much debt, that the family just dump it or try to recoup these new costs from anyone trying to buy the site.

I doubt anyone downloading right now will be looking to download from a mirror, as there are no official mirrors and presumably if they wanted to save brickshelf, they would go direct to brickshelf.

This is a problem. Even though people often refer to it as a community, it isn't really. There are many individual LEGO communities. There is no central hub where someone can say we are going to do this or that, and nobody else needs to do anything as it is being done.

[dr_spock]

5 hours ago, BrickWild said:

We understand that Flickr, ImageShack, PhotoBucket and other such similar sites are meant for Hosting, not Long-Term Storage.

As brilliant as that sounds, one of the drawbacks of having resources held hostage is that it ultimately isn't ours anymore.
Back in 2021, Flickr suddenly deleted my account without any statement but I was very lucky to get it back after briefly explaining why their claim was false.

I'm on the free 1000 image limit on Flickr.  I used to post the best pic of the MOC to Flickr, the other pics of the MOC to Brickshelf, and miscellaneous to Photobucket.  Photobucket pics are no more.  Brickshelf is going.  At this rate, I may have to setup a server at home. I don't think bandwidth would be an issue since I resized all my photos to Eurobricks specs of 1024x768 or 800x600 and  I am not popular. 

[aFrInaTi0n]

@MAB I totally agree on the risk of such scenarios creating unforseeable costs started by best intentions from the community - as I need to count me in there. It may be a very special situation for the old owner passing away, nethertheless back then when he owned the service he should ideally assured he is covered by costs in his own interest. I don't want to retrospectivly give stupid advises - but not doing so just allows for the risk being a possibility for the people having to solve the issues. So I would hope that he made sure this is the case.

This may also be a recommendation for people thinking of mirroring it on their own and directly mentioning in a second sentence that it may blow their own budget. If one has this direct fear coming up or can't assure enough security to have own budgets covered - then better not think further to not create yourself unforseeable harm - even if the intention have been the best, nobody is challenging anything else!

Just an example of being suprised: Who would have guessed that 4.2 million thumbnails in 300x300 max resolution would already count up to 227 additional GBs to be stored on disks - quite unexpected from my personal expectations and may be the same category of "better don't underestimate the internet" / at least for the thumbnails it was eyes-opening for myself.. And the zipfiles still need to be created.. :D

[R0Sch]

@SNIPE You clearly haven't used it much or you wouldn't say it was "horrible to use". And why put everything on Rebrickable? Not everything on Brickshelf are MOC's. There are a lot of set builds, reverse engineered models, renderings etc.
The most practical thing with Brickshelf for me was the ability to maintain the folder structure even if you'd delete and re-upload a new file with the same name, maintaining the link that was posted here in the forums. That way I could work on a Studio file or rendering and update it and users would still be able to access the latest version. Now I am left in the dry because Bricksafe won't accept new accounts. :((
@aFrInaTi0n Any chance to scrape also other type of files besides .png from Brickshelf? e.g. .ldr, .ldd, .dat, .mpd, .io, etc.? There are tons of digital models of MOCs which are much more useful than a bunch of shaky low res images in dark rooms. I also found .svg files of vector images from old theme logos from @oleth. It would be a shame to loose those too.

Edited February 3 by R0Sch

[aFrInaTi0n]

Yep, have all files, also non picture formats, so .io .ldr etc also covered.