Xwebarchives.org update

U

User1

RETIRED Admin, pm OFF
Just a brief tech note on the split personality "Daddy Xweb" we have been hosting in a holding pattern since my "semiretirement" a few months ago.

As some of you may recall, much fanfare was made by me about my/our efforts to get a functional mirror archive of N54 online on our own server such that we could use a more competent search engine to index it all? Well..

That never worked out. We had 2 main options, google, or a hosted solution (we looked at several and settled finally on mnogosearch). Well google is worth crap in free mode it is worse than the original N54 useless search unless we pay google $2000! to index it all (riiight)... and we were as it turns out actually in violation of some arcane policy of the host to be running mnogosearch, which the way the host server is configured was a tremendous pita to even try to get running, bottom line folks it's not panning out...

My "recommendation" to the 'guys' on this one is that I take the "1p0" mirror down, we reconfig N54 to be "the one and only 1.0 Xweb" archive as it had always been before, and all that ripping and mass file editing I did like a madman for the 1p0 effort, I will rewrite some of the links again using mfe as time permits, and Xweb can make it available on data sticks or something via snail mail order to any interested users instead of just continuing to host the useless search-dysfunctional mirror online with no forthcoming apparent use for it :hmm:

Incidentally, this frees up the archive server to be either discarded (if we plan no other use for it) or put to a new use (mentioned possible hosting alternative in the wiki thread)

As time permits I am going to be crusing around the N54/1p0 setup to "unlink" some stuff there and revert N54 to the archive pole position once more...
 
N54 restored

N54 has now been restored to "pre accident condition" :woot:

Xweb 1.0 :)
Enter the Wayback Machine and visit our original N54 DF archive
 
Tech to English Babelfish?

Could someone please translate this to English?

I'm not a total chimp but I'm not in the computer biz and I haven't been paying attention, so just tryin' to figure out the short and long term prospects for the archives.

There's a huge amount of awesome info in there that I hope is backed up elsewhere. I for one would be happy to pony up for one or more data DVDs containing the old archives, although I'm afraid they'd be of limited use unless they were searchable or otherwise indexed. Bein' a hardware guy instead of a software guy, I'm at a loss as to whether that'd even be possible.

But I digest (Family Guy Blue Harvest reference). Please allay my fears and tell me that there will always and forever be an Xweb archives, at the very least.

///Mike
 
The "Main Goal" was met

Plain english:

The "problem" we had was that N54 held the only copy of the full 1.0 archives, and despite the fact they themselves have multiple backups at all times, it still caused me some stress that "we" had none, in the event N54 ever went out of business, or got nuked etc.

So our "main" goal was just to get our own copy of the archives, then as a "bonus" we wanted to also host that online for you all to access.

The "attempt" is we did a full archive "rip" and then put that rip online with a (attempted) custom installed search engine. This would not only meet the goal of having our own full copy of the archives but also give a chance to stick a "real" search feature on it.

The "result" is that due to the sheer size of the rip (hundreds of thousands of files totaling over 2+gigs of data) we encountered problems. Due also to some host issues we were ultimately unable to get a custom search running on it (on our budget anyway, we would either had to spend way more $ to get the search going, or spend way more $ going for a more functional host, or both...), so in the end, we have decided to scrap the online mirror and revert users (you) back to N54 itself as (on our budget) that is the most sensible action.

The "met" part of the goal though is that, because of the above failed effort to host a rip online, we *DID* come away at least, with a full copy of the whole archive in our own hands now, just not online, it is sitting on my hard drive, and we will offer it for offline distribution to users at some future point, where a user could stick it in their pc or copy it off to their hdd, and although it is all static (has no db, no search) it would easily be searchable using a free tool like Google's "desktop search" or similar free hdd search tool (even Windows stock explorer search might do although due to size it might be very slow).

So the bottom line is, even though our effort to host an online mirror failed, we did succeed in making a full offline copy of the archive, so, even if N54 vanished one day, and took our data with them, we would still have it all, and at that point, would probably take another whack at putting it online ourselves. In the meantime it is just a lot cheaper to continue letting N54 do it.

I hope this clarifies things? :)
 
Other possible alternatives

There are also a couple of parallel concepts going on right now toward similar ends...

For one thing, right now there exists on Son Of Xweb a "partial" mirror already of N54 archive data, it goes all the way from the beginning up to just before 9/11, if you search for any old N54 thread in that range -using the SON OF Search- you should find a "thread view" (as opposed to post tree view) copy of that thread already living in the Son Of database.

Now that the online rip mirror has not panned out as intended, we could go back to this alternate project that was suspended during the rip effort, as time and volunteer labor permit. It is only thread views so it does not look the same as N54 did in post tree view, but all the data is here and fully searchable using Son Of db search. The "hidden forum section" which contains this stuff was in the process also of being subdivided into "tech" and "nontech" topics... http://xwebforums.org/forumdisplay.php?f=22

To complete this partial existing project would be a ton of work, but could be worth pursuing so the whole archive could be mirrored and searchable on our new forum...

The other thing, I am thinking since the 1p0 mirror is dead, we are rather likely to take it down and kill off that particular server so we can divert our donation kitty resources toward an oncoming wiki project (there is another thread on that elsewhere in DF), but, if we were to leave it up for a shorter term, possibly it could be used as a collaborative tool for rip error correction (the ripped archive is 95+% complete per my best guesstimate but it does contain a significant number of errored pages during the rip, under 5% but with that many files, that is still a noticable number in use).

We could assemble a volunteer team to manually scour through it looking for error pages from the rip, that we could then correct before making offline distribution copies. Not sure on its feasability or likelihood, since it would be another ton of work... just another random possible idea... For now, anyway, the defunct "1p0" archive is still up, here http://xwebarchives.org/1p0/index.php
 
Yes!

The tech-guy to English translation worked perfectly. Thanks for taking the time.

Dunno what effort is involved in updating/improving the archives as you suggest but let me know if I can help. Probably shouldn't offer with all the unfinished projects I have on the table but it'd be only fair to give a few hours to this great forum.

///Mike
 
You must log in or register to reply here.
Back
Top