Incoming Messages to Data Stream
The OTC market relies on a message scraping function called ‘IMGR’ inside of the Bloomberg terminal. In theory, this function scans the messages that come into your inbox and produces a list of securities for you to view – a quick way to see everything that may be available to you at any given moment. Bloomberg likes to present this feature as magic. In reality, the feature is highly a unreliable source of information, misidentifying or missing securities, and also adding phantom items to the list. Using the Bloomberg collection feed proved too incomplete, too unreliable, and, from a business perspective, too risky.
After review, it was found that the majority of the incoming messages had some sort of structure to them. While non-uniform across messages, the structure of the data inside of the individual messages allowed for a flexible internal search. Once properly indexed, relevant data could be sourced directly from the secondary parties, rather than relying on an intermediary like Bloomberg. Overall, collection and identification of securities improved by more than twice that of the IMGR process while also alleviating the reliance on unreliable third party software.
While data collection is at a reliable point, garbage identification continues to require refinement. Lack of internal resources prevents the process from silent operation and the live steam ultimately intended for the project.
Municipal Bond Location Identification
Production model will run on AngularJS to allow URL-based filters to allow preset, ‘shareable’ maps, among other improvements. Production versions will also incorporate relevant US Census and Wikipedia information.