Active Data Online Website Menu

Search Engine


 

Description

Active Data Online Search Engine is a hypertextual search engine, comprising a Windows Forms crawler, indexer and management console, and an ASP.NET search engine.

The system can crawl, index and store thousands of web pages, and return search results quickly.

It supports most languages. The crawler can fetch web pages encoded in alphabets other than Western Latin, and the data store can accept words in any language supported by UTF-8.

Search Engine uses its own data repository rather than a relational database server. Its file and data structures have been designed to provide fast access and keep operational costs as low as possible.

Search Engine is powered by Microsoft .NET version 3.5.

Page Menu


 

Key benefits

Why choose Active Data Online Search Engine?

  • It uses purposefully designed data and file structures. It does not rely on a relational database (RDBMS). Storing an index of tens of thousands of web pages in a database server such as SQL Server has a higher TCO than storing the same information on disk.
  • It clearly outperforms and outscales XML and text file based search engines. Its disk seek and indexing algorithms are based on ideas used by RDBMS's. An XML or a text file based system would require the entire data set to be loaded into memory before a search could be carried out, and thus it would be limited in size.
  • It is fast. Generally, results are returned within 1 or 2 seconds. The data structures have been designed especially for searching vast amounts of information quickly.
  • It supports most alphabets and page encodings. The system can detect page encoding automatically and adjust itself. There is no need for an administrator to micromanage the crawling and indexing processes.
  • It provides access to the entire result set. Many solutions restrict results to the top 1000 or so pages rather than allow the user to go through the entire result set for the sake of performance.
  • Human-computer interaction (HCI) experts say, a search engine that finds information accurately and quickly will enhance the usability of your website.

Page Menu


 

Current version

Version: 5.0

Release: 26 February 2008

Note: If you are an existing customer you may be entitled to the current version free of charge, per Active Data Online's version policy.

Page Menu


 

Flash videos

Page Menu


 

Demonstration

Note: Search Engine is designed to be a website search engine rather than a large scale google-like search engine. Whilst there is no theoretical limit on the number of pages, as more and more pages are indexed, for the average computer the system takes longer to return results. It is recommended that you use the fastest processor that you can. A reasonably sized website running on an average computer with hundreds or even thousands of pages should be able to return results in 1 second or less. On average, searching an index of, say, 10,000 pages for a common term such as "the" should return results in around 1 or 2 seconds, notwithstanding asp.net JIT compilation issues.

Page Menu


 

Features

  • The system is a search engine of hypertextual documents, comprising a data management Windows Forms application, which includes the crawler and indexer, and the asp.net 3.5 powered search engine.
  • Search Engine is ideal for websites with thousands of web pages.
  • Search Engine uses its own data store instead of a relational database server such as SQL Server or Oracle. This means it is possible to create and provide a search engine without having to purchase an expensive database licence or rent database server space.
  • The data store comprises of 3 binary files, suffixed .sed.
  • The processes of crawling and indexing are independent.
  • The URL store is a stack or in other words a LIFO queue. This means a crawler that needs to work through several websites will complete one website before moving on to the next.
  • It is possible to instruct the crawler to not follow any links, to follow links within the same domain as the current page, or to follow all links.
  • Relative hyperlinks (e.g., href="../page.html") can be handled by the crawler.
  • By design, the crawler will avoid meta tags, code comments, markup code, and the like - that is, all non visible text - when parsing pages.
  • The crawler will ignore blocks within sets of <!--stopindex--> and <!--startindex--> tags.
  • The crawler will not follow the pages of a HTML frameset.
  • The crawler will not follow javascript window.open() links.
  • As per W3C standards, Search Engine's crawler will obey "noindex" and "nofollow" meta commands within web pages.
  • Once web pages are in Search Engine's data store, they can be easily re-added to the crawling or indexing lists via the management console. As a result, a Search Engine can refresh its content on a periodic basis without the administrator having to retype all the entry-point URLs.
  • The management console provides an administrator with access to the data of a Search Engine - the URL crawling list, the page indexing list, the page data store, and the search index.
  • The asp.net search engine is encapsulated in an ascx control. For this reason, the search results page can be customise or embedded into an existing asp.net website.
  • By default, Search Engine supports most European languages - eg, English, French, German, Swedish, Spanish, etc. The system will detect page encoding automatically and adjust itself.
  • Searches are not case sensitive and punctuation is ignored.
  • Search Engine will only return pages that contain all search keywords.
  • Search Engine's page ranking system is based on the location of keyword matches.
  • The system uses data paging to handle large result sets. Unlike many Internet search engines that restrict results to the top 1000 or so pages, the entire result set is accessible.
  • Search results feature text preview and keyword highlighting. The results can be configured via appsettings to either return highlighted texts or the first few words.
  • Keywords within page titles are also highlighted.
  • The length of page titles (and URLs) in the search results can be restricted.
  • Searches can be easily logged so that it is possible to track what people are searching for.
  • In general, Search Engine can query the data store and return results in around 1 second. This is made possible by the proprietary data store and index. The data and file structures of the system have been designed from the ground up specifically for the purpose of searching.

Page Menu


 

Languages

English and other languages supported by Unicode. The GUI is localised so it can be presented in languages other than English, and the database can accept text in any language supported by Unicode.

Page Menu


 

Product support

Registered customers may seek support via the online forum, free of charge.

If you discover any bugs or the like with the product, please report it immediately so that the product can be fixed as soon as possible and an upgrade can be provided to you, free of charge.

Page Menu


 

System requirements

  1. Computer online and with Microsoft .net framework version 3.5 installed for running the management console
  2. Internet or intranet domain
  3. Internet Information Server with Microsoft .net framework version 3.5 installed
  4. Read/Write access to asp.net's App_Data folder
  5. Standard Web browsers such as Internet Explorer, Opera, or FireFox

Page Menu


 

Trust level setting

Medium trust level

Page Menu


 

Pricing and licensing

End User Licence, subject to the terms of the Licence Agreement

  • 150 Australian dollars

Notes:

1. Australian registered companies and persons residing in Australia must pay a 10% GST in accordance with Australian tax laws. The price with GST is 165 AUD.

2. If you are an existing customer, and you have the same major version number as the current release, you can receive the current version free of charge. Contact Active Data Online via the messenging system of the customer support discussion board to receive your free upgrade. If you are an existing customer, and you have a different major version number from the current release, you can upgrade to the current version for 50% off the list price. In this case, you have to submit an order but with the price halved. Please send a message regarding the upgrade order before mailing it.

Page Menu


 

Order

Before submitting an order, please read the system requirements and the licence carefully.

Each licence is for one server. You must specify a website domain, which can be a full domain (eg, yourdomain.com) or a subdomain (eg, search.yourdomain.com), and optionally the IP address of the server. Your licence will not work at another website.

Search Engine can be set up to run in the application root space of your website or in a subfolder of the application root space of your website. If you would like to run Search Engine in a subfolder, you must specify the domain (or subdomain, if it is a subdomain) and path to the subfolder (eg, yourdomain.com/search/). Your licence will not work in another subfolder or elsewhere. If you would like to run Search Engine in the application root, you only have to specify the domain or subdomain.

If you buy a licence for "yourdomain.com/search/1" and the IP address "123.456.789.0", then your licence will only work in the following locations:
http://www.yourdomain.com/search/1/SE_Search.aspx
http://yourdomain.com/search/1/SE_Search.aspx
http://123.456.789.0/search/1/SE_Search.aspx
http://localhost/search/1/SE_Search.aspx

Both http and https will work.

To order, fill out an order form (print it out via WordPad or IE7), one form per item, sign the form, and send the form together with payment to

Active Data Online Pty Ltd
P.O. Box 399
Sebastopol
Victoria 3356
Australia

Payment must be by bank cheque denominated in Australian dollars, made payable to "Active Data Online Pty Ltd ABN 83 095 152 453". (Important: personal cheques, money orders, cash and other forms of payments won't be accepted.)

Please ensure that your domain, path and (optional) IP are written correctly on the order form.

A sale is final and non-refundable.

If you are ordering from outside Australia, postage will normally take a fortnight or thereabouts. Please send a message via the the messenging system of the customer support discussion board about the order soon after sending it so that when it arrives it can be processed as quickly as possible.

Once everything clears, an email containing your download information and password will be sent to you, and you can proceed to download the product(s) via www.activedataonline.com.au. The package will include a tax invoice.

If your order is not approved, your cheque will be discarded and you will be notified by email. In this case, you should probably also cancel the cheque at your bank.

If you have any questions or concerns, please use the messenging system.

Page Menu


 

Roadmap

No plans in the near future, but as new ideas in search and AI emerge, and as hardware improves, the data structures and algorithms may be changed.

Page Menu




Copyright © 2008 Active Data Online Pty Ltd  ABN 83 095 152 453  All Rights Reserved