When will be able to search the BODY TEXT of the forums?
by Jarrod Roberson · in General Discussion · 05/14/2002 (10:33 am) · 17 replies
With about 35,000 posts most of which have titles that are not very descriptive to the contents, it is impossible to find what you are actually looking for, lots of the "old timers" are posting "search for it" since they have answered the same tired questions dozens of times.
as an example search the forums for "player speed"!
I know there are a couple of threads that talk about how to adjust player speed, what the default is set to and other things. I just happened to stumble across them. But to find them again, well . . . good luck.
as an example search the forums for "player speed"!
I know there are a couple of threads that talk about how to adjust player speed, what the default is set to and other things. I just happened to stumble across them. But to find them again, well . . . good luck.
About the author
#2
-David
Samu Games
05/14/2002 (10:43 am)
I'll second this...though I seem to recall a rumor that just such a feature was on the way.-David
Samu Games
#3
but it Kilt the server so they took it out..
I propose that the body search feature be used for only the SDK portion of the site and as well only for sdk users,
Then if this still too crunchy ..
limitations can be placed on the search ..
so you can only search once every 5 minutes or something
shrug....
05/14/2002 (5:20 pm)
I remember hearing how this usta be the way..but it Kilt the server so they took it out..
I propose that the body search feature be used for only the SDK portion of the site and as well only for sdk users,
Then if this still too crunchy ..
limitations can be placed on the search ..
so you can only search once every 5 minutes or something
shrug....
#4
Right now, you can restrict searches to a single domain or site in Google (see their advanced search page), and that does work with GG, except it does not "see" the forums, ie it only return pages from the public part of the website, etc. (like if you do a search for games, restricted to GG, it'll return quite a lot of results, but none in the forums)
It might be worth it to explore the possibility of Google providing search services, while still retaining the private part of the forum invisible to the public at large...
Just an idea,
Have a good one
05/26/2002 (2:46 pm)
...Or maybe find a way for Google to access the forum's content, at least the public ones ??Right now, you can restrict searches to a single domain or site in Google (see their advanced search page), and that does work with GG, except it does not "see" the forums, ie it only return pages from the public part of the website, etc. (like if you do a search for games, restricted to GG, it'll return quite a lot of results, but none in the forums)
It might be worth it to explore the possibility of Google providing search services, while still retaining the private part of the forum invisible to the public at large...
Just an idea,
Have a good one
#5
There were two solutions. Implement some half-baked, homegrown indexing, or just wait a little for MySQL's FULLTEXT indexing to mature.
Well FULLTEXT is finally in the stable build of MySQL even though it is a rather odd implementation, but it will work for what we need here on the forums.
I am currently working on a web based task/bug tracking system for Realm Wars and the Torque. After which I will come back and address some site issues like this one.
Thanks for your patience,
--Rick
05/28/2002 (12:24 pm)
Back in the day we had body searching using MySQL's LIKE text matching, which performs a brute force text matching. With the rate that you all post here (ha ha) the brute force method quickly started taking several seconds and all our server CPU for each query. Yikes! So it was disabled.There were two solutions. Implement some half-baked, homegrown indexing, or just wait a little for MySQL's FULLTEXT indexing to mature.
Well FULLTEXT is finally in the stable build of MySQL even though it is a rather odd implementation, but it will work for what we need here on the forums.
I am currently working on a web based task/bug tracking system for Realm Wars and the Torque. After which I will come back and address some site issues like this one.
Thanks for your patience,
--Rick
#6
Keyword extraction can be done by filtering the body text against the engine classnames and function names because thats what people will be discussing usually. The filter can be expanded to include a few extra words as needed.
In this way you can extract 10-20 usefull key words per body which can be searched in place of the full body text. It's not a full text search for sure but it should be almost as good and many times faster.
01/23/2004 (10:47 pm)
It should be possible to extract key words from the body text on the fly as a mesage is posted to the server.Keyword extraction can be done by filtering the body text against the engine classnames and function names because thats what people will be discussing usually. The filter can be expanded to include a few extra words as needed.
In this way you can extract 10-20 usefull key words per body which can be searched in place of the full body text. It's not a full text search for sure but it should be almost as good and many times faster.
#7
// Downloadable .zip by topic?
// I.E.
// Tech_AI.zip
// Tech_General.zip
// Art_3D.zip
// etc, etc..
This way those of us who are willing to can waste our own CPU cycles for the searches. :)
.
01/23/2004 (11:03 pm)
If nothing else for now, how about a "dump" of the public forums?// Downloadable .zip by topic?
// I.E.
// Tech_AI.zip
// Tech_General.zip
// Art_3D.zip
// etc, etc..
This way those of us who are willing to can waste our own CPU cycles for the searches. :)
.
#8
01/25/2004 (8:14 pm)
That would be excellent if we could get that.. I bet theres a trearure trove of info buried in the forums thats otherwise not finding its way to people who could use it :P
#9
As GG's server hardware is upgraded and more maintenance goes into the site, I think this feature will eventually resurface.
01/25/2004 (9:13 pm)
It would truly rock out. I know it's something Rick wants to do - I also think it's unlikely he'll release the db dumps you suggest, unfortunately.As GG's server hardware is upgraded and more maintenance goes into the site, I think this feature will eventually resurface.
#10
01/26/2004 (5:12 pm)
Yes PLEASE PLEASE PLEASE.
#11
wizard needs information BADLY
-s
01/27/2004 (3:15 pm)
How about anyone who wants to do a full text search gets bumped down to the lowest priority? I wouldnt mind waiting a minute for a search... wizard needs information BADLY
-s
#12
And 1-3 mins for a single category forum search is OK as well.. I.E. Tech>AI
// When I'm searching for info on "script" and "AI", I don't need to search the Orbz/Marble/TT forums and waste CPU cycles.
01/27/2004 (4:19 pm)
I personally wouldn't mind waiting 5 - 7 mins for a FULL (All forums) forum search as long as I could search within the results..And 1-3 mins for a single category forum search is OK as well.. I.E. Tech>AI
// When I'm searching for info on "script" and "AI", I don't need to search the Orbz/Marble/TT forums and waste CPU cycles.
#13
And Rick is out of town at the moment, anyway. I don't think he's due back until later this week.
01/27/2004 (8:43 pm)
Eventually, guys. :)And Rick is out of town at the moment, anyway. I don't think he's due back until later this week.
#14
// You're letting the programmers out of their cages and didn't
// warn us so we could lock up our daughters, pizza and caffeine?
01/28/2004 (2:18 am)
You let a programmer escape? What were you thinking Ben? // You're letting the programmers out of their cages and didn't
// warn us so we could lock up our daughters, pizza and caffeine?
#15
I really dont care about layout etc., so that would secondary for me. But to search full text is a must these days.
Example of free search engines I have used before myself:
http://www.aspseek.org/ (seems dead though)
But its only a matter of googling a bit and/or visiting freshmeat, and there is search engine galore.
Be smart - dont reinvent the wheel
01/29/2004 (12:16 am)
Ehhmmmm - why code it yourself? There are tons of good small (and free) search engines out there. Its basically install, point it at a start URL and let it index the site.I really dont care about layout etc., so that would secondary for me. But to search full text is a must these days.
Example of free search engines I have used before myself:
http://www.aspseek.org/ (seems dead though)
But its only a matter of googling a bit and/or visiting freshmeat, and there is search engine galore.
Be smart - dont reinvent the wheel
#16
tone
02/03/2004 (7:34 am)
The one easy wrinkle I'd like to see (in addition to the desire stated above) is to show me more than 10 or so subjects on a page. How about upping it to 20, 25 or 40? Would reduce senseless paging around and (I'm sure) reduce the HTTP volume on your site.tone
#17
02/03/2004 (7:45 am)
Check your options ;) 
Torque Owner Prairie Games
Prairie Games, Inc.
-J