IXSSO.Query - Ranking Formula???

[George Thompson]

I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).

My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.

As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.

Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.

If this is not possible, is there another option other than
IXSSO.Query?

Thanks!

[Hilary Cotter]

You can't override the default indexing services algorithms. You could
return all results and then apply your own algorithm to them.
--
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html

Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!

[Gang_Warily]

Hi George

There is a SQL Query Language for Indexing Service.
http://msdn2.microsoft.com/en-us/library/ms690505.aspx
it may not be as fast or reliable ...

The SQL extensions for Indexing Service do not support cross-data source
queries. You need to issue separate queries to another source — for example,
the ODBC Provider to Microsoft SQL Server™ — when combining information from
more than one source.

You may have to do some renaming of catalog properties to match database
fieldnames
if you want to use exactly the same query for database & IS.
http://msdn2.microsoft.com/en-us/library/ms691661.aspx
AdoCommand.CommandText = "SET PROPERTYNAME
'd1b5d3f0-c0b3-11cf-9a92-00a0c908dbf1' PROPID 'category' AS categoryname"
AdoCommand.Execute()

If you use full-text search on the database the SQL can look similar.

Not sure about ranking - I guess you could use similar vector/weighted
queries on both data sources ?
http://msdn2.microsoft.com/en-us/library/aa172868(SQL.80).aspx
http://msdn2.microsoft.com/en-us/library/ms691971.aspx#_idxs_weighted_match
Those are for weighting search words, but you wanted to weight different
fields/properties instead ?
I think you have to do that yourself, combining fields
... WHERE CONTAINS(Title...) OR CONTAINS(Keywords...) ...

I have used a SELECT COUNT(*) query to return just the number of hits for
each word in each field and build an optimised query to return the actual
result recordset using different weighting for each field/property.

Lots of queries that just return a number is very quick,
but I'm haven't got very smart in using the 'which words are in which field'
results.
It may be just as quick to query every word in every field.
I wanted to return just the 100 best hits, avoiding searches that return
1000s !

Sending similar weighted queries to both database & catalog should give two
recordsets that could be merged ?

It isn't easy, though !

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!

[Gang_Warily]

Hi

Just found
http://msdn2.microsoft.com/en-us/library/ms345119.aspx
The science of ranking is far from mature, and as the field evolves,
Microsoft may change how ranking works. Therefore, applications built on
MSSearch must not rely on any particular ranking implementation, or they may
break when a new version of MSSearch is released. In fact, ranking in
MSSearch of 2005 differs in several ways from ranking in products such as
Sharepoint Portal Server, Indexing Service, SQL Server 2000, and others.

Definitely implies do your own simple ranking scheme if it must be
consistent between Indexing Service & SQL Server !

http://msdn2.microsoft.com/en-us/library/ms948065.aspx
gives some idea of the different ways ...

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!

[Gang_Warily]

Hi again

You might be better to approach it from the SQL server side:
http://msdn2.microsoft.com/en-us/library/aa172902(SQL.80).aspx
Full-text Querying of File Data
example of SQL Server 2000 querying "OLE DB Provider for Microsoft Indexing
Service"

Equivalent example not given for SQL Server 2005, but
http://msdn2.microsoft.com/en-us/library/ms190479.aspx
looks like a stored procedure to create the linked server in SQL Server 2005.

I would guess you could put both of those together and query the file search
catalog together with the full-text database search ?

Let us know how you get on !

Eric

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!

[Gang_Warily]

A slightly lateral-thinking approach would be to publish all records of the
database as files on hard disk, then you could just use Indexing Service to
search those !

You can add custom meta tags to html files.

It's not that crazy - some top Content Management Systems do just that:
Percussion Rhythmy
http://www.percussion.com/products/content-management/rhythmyx/downloads/generic-2213.html

It might help performance if you just webserve a file instead of building
pages on-the-fly from the database.

It also can give resilience against database problems,
and simplify web-hosting and site-mirroring ...

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!

[Gang_Warily]

A slightly lateral-thinking approach would be to publish all records of the
database as files on hard disk, then you could just use Indexing Service to
search those !

You can add custom meta tags to html files.

It's not that crazy - some top Content Management Systems do just that:
Percussion Rhythmy
http://www.percussion.com/products/content-management/rhythmyx/downloads/generic-2213.html

It might help performance if you just webserve a file instead of building
pages on-the-fly from the database.

It also can give resilience against database problems,
and simplify web-hosting and site-mirroring ...

Post by George Thompson
I am writing a search engine that will search a database for matches,
using a specific ranking formula (the user is given the option of
weighing things like the title, meta data, body etc etc to affect the
results).
My problem is that it also search files (doc, pdf etc etc). I have
reviewed IXSSO.Query, and can't seem to find a way of overriding the
default ranking system and adding my own ranking system that matches
the one used on the database records.
As it stands now, there is no ranking relevancy between files & db
items that are being returned on a search.
Ideally, they could both be displayed together in one list (by
ranking), but that will only happen if I can override this ranking so
that they use the same formula.
If this is not possible, is there another option other than
IXSSO.Query?
Thanks!