Dryw Filtiarn

I don't know if a topic like this already exists, if so, feel free to just close this topic 8-| Anyway, what are you currently working on. My main target then is Lego MOC's ofcourse. I'm currently preparing for the LegoWorld event in the Netherlands (october 18th to october 23th), the thing I'm working on to hopefully be finished in time to show of is a Technic supercar based on the Mercedes CLK GTR, which is probably best known as the Le Mans 24 hour GT car, which took of and flew over the fence. What I have finished by now is the front suspension and the rear suspension. And currently I'm working on the part in between, which basicly means the gearbox and the chassis to hold the gearbox and suspension parts together. EDIT Bonaparte: Merged!

Not a buy today, but 2 days ago. The 8420 Technic Motorcycle. It was on clearance sale, nearly half the price it used to be *sweet*

Still want to try it, but in general I forget to do it, before I tear open the bag 8-| It's just a shame that this can't be done with bigger sets though.

I'm quite sure that he has been blocking my IP address previously (as mentioned, it's unblocked again) as I simply could not get onto Brickshelf nor Maj.com, while at the same time I could access them from elsewhere.

Have you been downloading heavily??? If so Kevin probably blocked your IP adres, just like he did to my main IP address, as well as 2 others hosts that I was using to proxy (for the fact that my main IP was blocked). I must say though, that my main IP already has been unblocked.

The person above me has a very nice avatar.

It's entirely random :) It's based on username as I grab on per-user bases. Edit: I just bought and installed a 500GB harddisk to occomodate for further downloading :)

Over 250.000 files have now been downloaded with a total size of 62.9GB and everything is still running :)

Update: I finally have reached more then 100.000 files. Currently the number of files downloaded is around 101.000 (these files belong to a total of 3821 people) with a total size of 22.5GB. Some more statistics regarding the average filesizes: Files < 1MB: ~97.17% of all files Files > 1MB and < 2MB: ~ 2% of all files Files > 2MB and < 3MB: ~ 0.68% of all files Files > 3MB and < 8MB: ~ 0.04% of all files Files > 8MB: ~ 0.01% of all files The overall average filesize calculated over de current 100.000 files downloaded is: 235.96kB The a bit of information on the speed of the whole process: In 1 hour the following gets processed on average: 17108 files are being downloaded contained in 2520 folders (galleries) belonging to 972 users with a total size of 3.83GB. Knowing all of these values will mean that the entire downloading process will take another 82 to 100 hours (which will be 3,5 to 4 days). The timeplan though, is based upon the fact that I won't get any interruption from Kevin blocking me again :'-)

Nah, before I run out of webhosts to host the proxy script on will take a while *sweet* And if I do run out before I have been able to get everything, then I'll simply find a new sollution, just like the sollution of the proxy script now *sweet* BTW, my scripts should also be less detectable now :'-) I'm now on: 77.000 files and 17.6GB (I could have been further if I would have made a stupid mistake corrupting some of the data :-$ ) Edit: According the the current progress the total data for all the photos currently on Brickshelf will be somewhere between 370GB and 413GB (based upon filesize averages)

I'm currently somewhere around 64.000 files and 14GB My processes are running again. I managed to play around a bit with some other webservers to create a file-proxy there, which allows me to access brickshelf :)

I've been getting to about 30.000 images, which is far from enough unfortunatly :(

We'll that's pretty easy :) In my effort to find more information about the BS issues, I came across the EB site :) Now I need the time to properly read through some topics... Yeah, part of my name is Welsh, the other part is old Irish :) (It's just my nick btw, not my real name ;-) ) @ The rest, thanks for you welcome :)

So far copying the data from BS. Kevin apparently decided to ban my IP on the server, meaning that I can no longer visit BS nor Maj.com. Edit: Typo

I'll keep you posted over the coming few days (as this will be a multi-day effort to be able to retrieve everything). Regarding my previous post, I might have to correct the amount of users that I had been able to find on BS. I'm currently rerunning the scan (doens't have any effect on the timeplan though!) to figure if I was correct. It appeared that there was an oddity in my script that scanned the useraccounts. Will get back on that issue.

I'm strictly monitoring the progress and averages on filesizes at the moment. The statistics I can currently give you are the following: At this point the average filesize over all files already downloaded is about 270kB. Which will result in about 420GB of data when everything is downloaded (theoreticly). I must say though, that I start to doubt the statistics on the site a bit regarding the 1.9 million shared files that would be on BS. The initial thing that I have been doing is retrieve all user accounts that exist on BS, and I can't get any further then a total of 34569 users, which I'm pretty sure of to be correct. When I compare that to the average amount of files per user (based upon the already downloaded content, this results in aprox. 1 million files, where I must admit that the following filetypes are being excluded: avi, mpg, mov, wmv, wma, mp3, ogg and for safety also: exe, com and pif). Anyway, everything is going pretty well right now :) BTW this whole operation is being done by selfwritten tooling using the command line, and it's currently running 10 parallel processes, which uses my available bandwidth pretty well (nearly maximum capacity).

Organizing the data is not much of a problem, that's just a matter of downloading the stuff as is. When I leave out huger files (like video's) you will probably not really be talking about TB's of data. Considering the fact that there are aprox. 1.9 million files on BS, of which probably 5% will be video (yes there are actually some folders which are nearly entirely packed with video), and of which aprox. 10% will be textfiles/ldraw files. Then we will be talking about the following amount of data: 1.9 million minus 5%: 1.8 million of which: 180.000 will be textfiles/ldraw (or other small text type files) and: 1.6 million will be images The average image size will somewhere in between 150 and 250KB (i'm talking AVERAGE here), the average file size for text/ldraw will be in between 5KB and 25KB (once again AVERAGE). Then I come to a total sum of: 1600000 * 200KB = 320000000KB / 1024 = 312500MB / 1024 =~ 305GB 180000 * 15KB = 2700000KB / 1024 =~ 2636MB / 1024 =~ 2.6GB So we are talking about 310GB + or - 100GB (to be safe) is somewhere between 210GB and 410GB. Which is pretty doable :)

I'm working on a full 100% coverage backup: Please also check this topic: Community Project: Retrieval of "Lost Images"

I'm currently working on storing everything that can be found on brickshelf. This means a 100% backup, so this way there will at least be more people beside the site owner having all the images. When it's done it will initially serve as a recovery backup. So people who did not make it to retrieve their image (i.e. due to holidays) can give me a message to get their stuff back. What is to be done with the content in the future is to be determined at this point. It's likely that video's will be excluded though for the fact that they take a long time to capture as well as that they take up a lot of storage space. And in my opinion it's the images and ldraw files which are of most importance.

You're partially true there. But what if the situation arises where only content is restored for people that actually pay and all other content is dropped or at least inaccessible. Then it might be an attractive alternative when there's a free site that offers the seem thing for free. And all the images that broke... yeah, that's a shame :(

I think that it if will actually become a fee site that some people will step up and create a free alternative. There is basicly no reason for making it a fee site. As it will probably raise enough money from people that are willing to donate as well as that it can raise considerable money from ads that are on the site. Considering the cost the site should easily be self paying. The only thing of concern is the cost of time. But if you would find a dedicated team of people to maintain the site it's pretty doable (and I'm sure that it's easy to find a couple of dozen people that are willing to help :)) Edit: Forgot a word, which made it unclear ...

Even if an replacement for Brickshelf is being found, the images will still remain dead, unless every single person updates every link he or she ever posted. In my opinion the only sollution is to take over Brickshelf from Kevin. And thats what Jennifer and Dan (Peeron.com) seem to be working on atm. Lets just wait for that. Maybe they will get an agreement of some sort in a couple of days and brickshelf will be reopened.

Personally I must say that I don't have much faith in maj.com at this moment. Something is telling me that it's a matter of time before maj.com will also be shut down. Apparently it has been happening earlier with this same guy that he shut down popular websites without any notice beforehand.

He's entirely right.. I am from planet earth. My current location is: Enschede (Overijssel (The Netherlands (Europe (The world (The univers))))) :-D

Got a bit of news which starts to clear up some things (in a negative way). Jennifer (Peeron.com) posted a message on the Bricklink forums that they were on top of it to find a sollution for this situation. Dan (Peeron.com) also said the same thing on lugnet. In my opinion this does say something about the whole situation (and about it being permanent, at least for now...)