Varda
Vala, Council
Posts: 1,041
|
Post by Varda on Jun 7, 2004 17:12:08 GMT -5
Our Tolkien Encyclopedia needs help from some of you nifty programmer folk. A reader requested a Search program that can look up any name in the Encyc, as it took him five minutes to locate the entry for "Huan". It seemed like a good idea.
If you need me to tack something onto the article names, or if they just need to be consistent, or you need some other form of help, please let me know.
|
|
|
Post by arathorn on Jun 7, 2004 20:09:00 GMT -5
Searching anything on the internet is ugly. To search the web pages themselves would be inefficient to say the least. Depending on how Farmer Maggot set up the pages he might be able to code it (He used a version of Basic if I'm not mistaken) to search data if he stored it a certain way. I suppose he could search the pages themselves, but extracting the data and reformatting it for redisplay could be a pain. The best is to have a database AND internet friendly setup to run such programs and store the data. Of course, we'd have to know what we have available (not all places that store pages allow that kind of access to run programs and store databases/files. I've written stores on the net in various languages using data files and databases, but to implement them I had to have access to the system files of the provider (was working for them at the time). This is hard to finagle without paying business fees, most likely I haven't been on much since my injury so my info/participation will be sporadic unfortunately, for the next few weeks. That's my 2 cents at least.
|
|
Varda
Vala, Council
Posts: 1,041
|
Post by Varda on Jun 7, 2004 22:41:17 GMT -5
This does not bode well.
|
|
|
Post by arathorn on Jun 8, 2004 1:32:59 GMT -5
If FM can grab a gander at this thread, he may shed more light on what we can do based on his setup.
|
|
Eonwë
Vala, Council
Vala and proud member of the Valar Guild. A quarter century of Tolkien fun.
Posts: 1,322
|
Post by Eonwë on Jun 8, 2004 2:10:46 GMT -5
Hmm,...
I assume we're talking about being able to find the name "Huan" in any article or paper on the Encyclopedia, although the example seems from the example we're only talking about article names. If it's the latter, I always thought the sections were clear enough myself, but that's me :} If it's the former,.. Searching for a name, word, or phrase sounds simple enough to program at first glance.. it's getting the program to search the entire encyclopedia page by page with minimal need to update (i.e., every time a new page is attached adding a file to a list for checking) and as Arathorn notes, the displaying the results of the search, as well as linking directly back to that very spot where the word, name, or phrase is located.
Writing such a program from scratch, in C++ or another language, would take time, and plenty of it, depending on the skill of the programmer. I have slight ideas where to start but not how to finish, to be honest :} I don't think I'd want to dig into such a program just yet either.
There probably would be ways to do it using a database program like Microsoft Access, but that requires some level of knowledge on that as well.
I agree with Arathorn. Farmer Maggot's already got the basics set up and probably more than that, so he's leaps and bounds ahead of anyone at this point :} ---
The Encyclopedia says he's got the search engine down for logic work. Once he's got the offline version working, I can't imagine it'd be too hard to convert it for online use, but then I'm optimistic :}
Take care :}
Hope your injury isn't too serious Arathorn.
Eonwë
|
|
|
Post by arathorn on Jun 8, 2004 11:33:57 GMT -5
A broken toe is a broken toe... just have to wait for it to heal.. sets your plans back for weeks... more of nuisance than anything. Back to the encyclopedia, I've written what you're talking about in both c++ and perl on linux. I've also coded in MS Access's visual basic language and in various other database languages (or even languages accessing straight data files). In any of the cases, obtaining the data is a snap with a database and then it has to be formatted into a dynamic html page which is sent back to the user. The coding/creation of this isn't all that difficult. The problem lies in the fact that the server (the Internet Provider) has to have the setup to allow the developer to implement the database and the programs. This is rarely provided because system files on the server have to be changed or created or even certain program languages installed for the application to work. This means they are giving you access to their operating system files and basically free run of their server (or at least enough of it to damage it whether accidentally or maliciously). Coding/implementing would take a bit but isn't really hard. Having access to the environment to put it on, there's the rub. You have to be able to test it on their system, make changes accordingly, etc. I've done it before, but then again, I was working for the ISP on contract, friends of mine, so had free run of their system ;D. It's fine to have all the plans and materials to build a house, but if the landlord won't let you dig the first hole ... well..
|
|
Varda
Vala, Council
Posts: 1,041
|
Post by Varda on Jun 9, 2004 0:12:45 GMT -5
Farmer suggested I ask you, Arathorn. The valarguild.org server belongs to Alatar-Valar. It really sounds like more trouble than it's worth. Imagine going to all that trouble then hearing the Encyc has just been changed to make each article on a separate page, as I plan in the future. Thanks for considering it, anyway. Let them spend five minutes hunting, I guess. : )
|
|
|
Post by arathorn on Jun 9, 2004 11:47:19 GMT -5
Yeah, I know that (Alatar's server). I tried running some simple perl scripts on there but they wouldn't work. My guess is that the web server needs to be tweaked to allow us to run scripts of that nature. I emailed him about it at one point but we never got anywhere with it. In any event, after talking to FM online, I've learned that the Encyclopedia he wrote is one that you download and run on your pc. THAT I did not know. I thought there was something interactive on the pages themselves, which I now know, there is not. Therefore to search the pages as they are would be a completely different animal. ;D Either an entire new web app would have to be coded (yet implementing it on Alatar's server could prove to be more work than its worth) or perhaps there is a facility that can be added to the site that allows for searching of pages and displaying results (kind of the way message boards or even 'Google' do)... I seem to remember something of that nature, but never pursued it. I'll take a look around and see if anything pops up. ... And if it does I'll knock it back down! ... SCAT!!! er.. that is all.
|
|
|
Post by FarmerMaggot on Jun 9, 2004 14:03:49 GMT -5
I've been following this thread, but haven't had a chance to reply yet. I'm rewriting the Encyc app again because I'm still not happy with it. I've been playing around with a different database system that should work better. I was thinking: 1.) Get the database working 2.) Get the search code working 3.) Split it into 2 projects: the downloadable application, and a webserving application -> the webserving application would provide the same/similiar interface as the downloadable application, but it would be accessable from the internet
The webserver application would run on my computer (or another computer that's always online).
I'm hoping I can find time this week either before or after finals to get some serious work done on this.
FarmerMaggot
|
|
|
Post by arathorn on Jun 9, 2004 14:08:05 GMT -5
Keep me posted. I'm familiar with all the aspects of what you are trying to do and could even help code/test it (most likely would run on my linux machine too, re: coding/testing).
|
|
|
Post by arathorn on Jun 9, 2004 15:07:35 GMT -5
Well, just as an FYI:
Took a look around and found a couple of programs that allow you to create a search page after running against your website. I downloaded the demos and tried them out and they're quite good. Of course there's a 20 page limit in the demo and you'd have to rerun the program whenever you made substantial changes to the site/pages, but in a mere matter of seconds it indexed all the words and created 3 files (1 html, and 2 javascript) that you would simply upload to the website and the search would work from that page.
One of them only costs $60 (to have no limit and other special features). Would definitely be worth it to any company requiring a search page for their site. I was quite impressed ;D
|
|
|
Post by FarmerMaggot on Jun 9, 2004 16:55:16 GMT -5
Arathorn - Let's move over to the Programming section and continue discussing the technical sides of this Joe
|
|
|
Post by arathorn on Jun 9, 2004 18:01:59 GMT -5
AAAWWWWW! We never even got to page 2!
|
|
Varda
Vala, Council
Posts: 1,041
|
Post by Varda on May 14, 2007 14:42:28 GMT -5
|
|