Psience Quest

Full Version: Alex has temporarily reactivated deleted forums
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3
disappear the plan... cutting a
Laird has a plan and has been conversing with Alex so I guess we'll know in good time. From what I understand (and I might be mistaken) Alex is not too motivated to provide a database export so other solutions might be required. As I say, people are aware.

Chris

Please do let us know whether there is anything we can do to help to preserve the material.
Ah, now I understand what 'CD' meant: Critical Discussion.
(2017-08-15, 11:27 PM)Max_B Wrote: [ -> ]
(2017-08-15, 11:11 PM)Kamarling Wrote: [ -> ]Laird has a plan and has been conversing with Alex so I guess we'll know in good time. From what I understand (and I might be mistaken) Alex is not too motivated to provide a database export so other solutions might be required. As I say, people are aware.

Thanks for the info, but this is about my posts, not the forum. I just don't want to duplicate my effort for no reason, if they are *sure they can get em*, if there is even the smallest risk they can't get/restore them, then I want to save em for myself.

Max, you can, if you are concerned, right now use my forum-scraping app, FUPS, to pull your posts and only your posts from the Skeptiko Forums (unfortunately, it doesn't allow you to limit the scraping to just CD and Other Stuff). I am in the process of extending this app to pull all posts in a (sub)forum out of a XenForo forum - FUPS already supports this for phpBB - and I am well into this (not particularly difficult) process.

Please let me know if you have any problems using FUPS.
(2017-08-16, 12:22 AM)Max_B Wrote: [ -> ]What are you entering for the 'Thread URL prefix' field?

That needs to be left at its default for Skeptiko.
(2017-08-16, 12:31 AM)Max_B Wrote: [ -> ]
(2017-08-16, 12:25 AM)Laird Wrote: [ -> ]That needs to be left at its default for Skeptiko.

I'm getting an error... wonder if that's because I don't exist... Max_b is no longer searchable, as I'm banned.

Argh, I'm sorry, Max - the problem is that member pages aren't visible on Skeptiko unless one is logged in. Right now I don't have a workaround for this - FUPS supports logging in for phpBB, and for phpBB it also supports specifying one's username so that FUPS doesn't have to try to look it up, but FUPS doesn't currently support either of these options for XenForo. The easiest one to implement would be to add a setting to the XenForo options to allow the stipulation of the poster's username, but unfortunately I'm in the middle of a bunch of other changes, so can't get onto it until later. Will let you know as soon as I've implemented this setting for you. Again, I'm sorry.
(2017-08-15, 11:52 PM)Laird Wrote: [ -> ]Max, you can, if you are concerned, right now use my forum-scraping app, FUPS, to pull your posts and only your posts from the Skeptiko Forums (unfortunately, it doesn't allow you to limit the scraping to just CD and Other Stuff). I am in the process of extending this app to pull all posts in a (sub)forum out of a XenForo forum - FUPS already supports this for phpBB - and I am well into this (not particularly difficult) process.

Please let me know if you have any problems using FUPS.

Laird,

Could you just get a copy of all of skeptiko (all users)? Would that be an easy change to your program? Could you restore/import it as a sub forum here? (Then you could delete the sub forums that Alex is retaining if you wanted.)

Do you know what is the format of backed up xenforo forums? Is it text / xml or binary? Could Alex back up the forums in question and then let us download the backup files? (All I want is something readable and searchable.)

Can myBB import xenforo backup files? Certain forum software can import forums from other types of forum software. Is there a migration path from xenForo to myBB?
(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ]Laird,

Could you just get a copy of all of skeptiko (all users)?

That wouldn't be a good way to go about it because there would be no thread continuity, just a bunch of collections of posts by individual users.

(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ]Would that be an easy change to your program?

I've almost finished the changes that will allow us to scrape all threads in a set of specified forums - so, will probably scrape CD, Other Stuff, Consciousness & Science, and Extended Consciousness & Spirituality. I actually have something functional, there are just a few bugs to iron out.

(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ]Could you restore/import it as a sub forum here?

That could be tricky. The posts will be in HTML, and I don't think MyBB (the forum software running PsienceQuest) supports raw HTML in posts. Maybe we could set some sort of static restoration up in the web root (I deliberately ran the forums under the /forums directory so that we have the option to do stuff like that).

(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ]Do you know what is the format of backed up xenforo forums? Is it text / xml  or binary?

As best I can tell, the only way to back up XenForo forums is to back up the underlying (MySQL?) database. The format would thus be SQL.

(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ]Could Alex back up the forums in question and then let us download the backup files?

I don't think it would be easy to provide a backup of just the forums in question, but it would be straightforward for him to back up the entire database and provide it to us. I PMed him asking him to do that very thing, but he didn't respond.

(2017-08-16, 03:39 AM)Jim_Smith Wrote: [ -> ](All I want is something readable and searchable.)

I think I'll be able to satisfy those criteria. :-)

One thing my script doesn't handle though is avatars. It will at least retain usernames.
Is it worth just sending a webcrawler to get the stuff from Critical Discussions?

I've grabbed at least a good chunk of the Resource Thread info, at least AFAICTell, from just saving the HTML of the pages, but I notice there are some good discussions that are worth preserving in the mix.

Ah didn't realize that Exploration is on the chopping block too? I'll try to comb through and grab Resource threads but not sure if I can get everything just by hunting & pecking.

I'll try to see if a cheap/free webcrawler can grab everything off at least CD & SE.

Laird if you're grabbing stuff reliably honestly I would just get stuff from Consciousness and Science as well though maybe wait until after the deadline for the others. No telling when any of that content might end up gone.
Pages: 1 2 3