One of the key reasons for choosing Symphony is to gather data at a speed that is much quicker than that possible if a human being was performing the collection. Not only is Symphony faster, it is also much more accurate, as soon a human being attempts to gather data faster, the accuracy of that data is likely to deteriorate. Using Symphony you get the best solution, accurate data, gathered much faster than using any other method, even when the amount of information involved runs into thousands of pages or records. Symphony is designed to ensure that data is gathered quickly but also responsibly, ensuring that there is no detrimental effect to the source websites.
Symphony is flexible to your requirements and data model, this means that we can ensure that the format of the data you get is to your specification, so that it is automatically compatible with whatever application you need to use it with. It is entirely possible for the data to be made available directly to your external web services, so that the removing any need to manual intervention and downloading.
By spreading the load across many machines we can not only maximise the performance of the service, but we can also guarantee its reliability, as part of our SLA we actively monitor the websites, when the sites design or specification changes, the team are alerted to this as part of a nightly build process. If the changes to the websites are not wholesale, then the configurations are fixed at no extra charge to the customer, the service continues seamlessly.
Configure a crawl schedule to suit your specific requirements, if we were running the crawl weekly then we could stagger the crawl to run different sources each day, and maybe the same source multiple times. This would mean that on each day we are doing less sources which means they will be finished quicker which means the data can be delivered in batches as and when certain website sources have completed.
Key benefits of this include:
Symphony can automatically complete online forms such as product searches or filters, to allow you to gather data from websites even in circumstances where normally a human being would have to manually complete a form. Say for example you need to enter many different types of car makes and postcodes into a website form and click the button to retrieve car hire details, Symphony can do this entirely automated, it can even gather the required makes and postcode information from other websites or databases and use this to perform the searches. This results in collection of the exact data you need in a format that makes analysis immediately possible without any additional data clean up.
Where similar data is being gathered from multiple website sources, Symphony’s data consolidation feature can combine the data into a single data set, identifying commonalities and using this to group and sort the complete data set, making for more efficient and accurate analysis of the information. Imagine a situation where you collect data from four different websites and need to produce one single set of data grouped by address and postcode, Symphony does this with ease saving you hours of manual work.
Sometimes when collecting, combining and analysing data from many different sources each set of data may use different terminology to refer to the same thing. As an example, carhirecompanya.com refer to it’s luxury range of vehicles using the term ‘Luxury’ where as carhirecompanyb.com prefer the term “Prestige”, what you need though, is all the luxury and prestige cars grouped together. Using Symphony we can provide a specific customer taxonomy file that makes grouping this data entirely possible.
Often when gathering data for analysis it is useful to be able to capture a screenshot of the web page at a point in time as this allows you to refer back to the original webpage and see exactly how the data looked in context. This is especially important in situations where legally you need to capture what is on the screen at a point in time. Symphony’s screenshot feature can take a screenshot of the page as it collects the data, even scrolling down the page to ensure your screenshot captures the entire page and not just the part displayed on your screen.
Where you require images from the source websites Symphony can be configured to download and provide the actual image files and where appropriate reference the related data. In addition Symphony can also resize the images dynamically, to make storage or transfer more efficient.
If the information you are gathering from websites includes prices for products or services. Symphony’s currency conversion feature means that it can be configured to perform live currency conversions on the data at the time the data is collected or at a later date. Our data will detect the native currency of the web site and then provide you the figure in any other currency along with the conversion rate we used in the calculation. Similarly if the data on the web site is in a mixture of kg and lbs but you want it to be provided to you already converted to grams Symphony can take care of this.
Symphony can be used to collect data from many different websites and therefore that data may be in very different languages. Symphony has language conversion built in and therefore can convert all the data to any language you require instantly.
If you are using Symphony to collect data from websites that involve UK towns and districts and these need to be converted to postcodes, then Symphony can handle this with ease, providing you with data that can then be used to search and filter by geographic location.
By GEO tagging your information as it is collected, Symphony enables the data to be plotted on a map, therefore making it possible to analyse the data in terms of where it is located, including calculating such useful information as distances and averages about regional trends. Say for example you wanted to know the average cost of products for a certain sector and wanted to know how this varies by where in the country it is located. Symphony makes all this possible.
Symphony has been designed with flexibility in mind, this means we can divert any of the major steps of the Symphony software to execute custom code written specifically for you, for those rare occasions when the core functionality isn’t quite right. This makes it possible for us to perform one off calculations, specific text manipulation or even looking up data in a database and including this in the final data set easily achievable.