Market regulator, the Securities and Exchange Board of India (Sebi), has reportedly decided to deploy a ‘web intelligence tool’ to increase surveillance of social media and other online platforms. The intention is to probe violations of securities laws by individuals, groups, and other entities.
The regulator has invited Expression of Interest (EOI) from solution providers to implement, install and maintain the tool. But what is ‘web intelligence’ and how does it help investigators?
What does Sebi mean by ‘web intelligence tool’?
In the public notice released on Monday, Sebi said that the tool should be able to gather unstructured publicly available data from the web, including public websites, social media platforms, and open source databases, among others.
The new tool should provide Artificial Intelligence (AI)-based solutions to extract and analyse unstructured publicly available data to gain deep intelligence insights about various entities, individuals, groups, and topics. It is expected to save time, ease the analysis process and improve efficiency in the overall investigation process.
Basically, the regulator expects the tool to enable investigators to easily analyse massive amounts of complex unstructured data using its AI and data analytics capabilities.
What are the requirements formulated by Sebi?
The ‘web intelligence tool’ should be able to visualise the data in the form of a network chart to find insights about connections among relevant entities, individuals, and groups. The visualisation needs to be interactive and must allow investigators to work with the data as per user requirements.
Sebi also needs the tool to generate ready-made reports based on the defined report templates. The regulator noted that users should be able to create and modify report templates without any coding requirement. Interested parties can submit the bids by October 3, the regulator said.
How does web intelligence work?
Web intelligence picks out only the relevant information from massive data gathered from the internet. This is done using a combination of technologies including AI, database management, Web science, Semantic Web, and information retrieval.
A series of tasks is executed for extraction, exploration, and utilisation of unstructured data available on the internet by using web mining techniques. The aim is to automatically discover and extract information from the resources on the internet.
Web intelligence uses the World Wide Web (WWW) as a phenomenon of retrieving information from data storage in an efficient manner. This web data is developed with the help of Semantic Web tools such as Resource Description Framework (RDF), Web Ontology Language (OWL), and SPARQL. One of the most widely used threat intelligence subtypes for the web is open source intelligence.
What are open source intelligence tools?
The intelligence derived from publically available data and information is known as open source intelligence (OSINT). Open source information is not limited to the data that can be accessed using the major search engines. Though the web pages and other resources that can be accessed from search engines constitute an important part of open source information, over 99 per cent of the internet cannot be accessed using the major search engines.
This part of the internet is represented by the “deep web”, which is a mass of websites, databases, and files that cannot be indexed by web search engines for reasons including the presence of login pages or paywalls. Government agencies including military departments, law enforcement agencies as well as cybersecurity organisations use OSINT for investigating and tracking cyber criminals.
What are the functionalities of web intelligence tools currently used by different organisations?
A large number of free as well as premium tools can be used for functions like metadata search, code search, people and identity investigation, for searching a phone number, email search, and verification, linking social media accounts, and image analysis.
The advanced tools can also help in wireless network detection and packet analysis as well as geospatial research and mapping.