Efficiently consolidate your environmental data
Chris De Cree, an EHS Implementation Manager from Locus Technologies walks us through our container tracking tool.
Tag Archive for: Data Quality
The Past, Present, and Future of EHS & ESG
To celebrate a milestone 25 years of success in EHS and ESG software development, we sat down with Locus President, Wes Hawthorne for a brief discussion. In this post, we ask him a series of questions highlighting the past, present and future of EHS and ESG.
1. What are the greatest challenges that environmental professionals are facing?
One of the persistent challenges we’ve seen for the past 25 years is that the responsibilities of environmental professionals are continually expanding. Previously, almost all environmental work was localized, with facility-level permits for air, water, waste, etc. That has expanded over the years to include new regulations and reporting requirements for sustainability, social metrics, and other new compliance areas, while the old facility-level programs still continue. This has led to more pressure on environmental managers to keep up with these programs, and increased reliance on tools to manage that information. That’s where Locus has always focused our effort, to make that ever-expanding workload more manageable with modern solutions.
2. What are the most interesting trends in EHS and ESG?
The current flood of interest in ESG is certainly notable as far as bringing corporate attention to the environmental field, as well as having requirements originate from the SEC here in the US. We have become accustomed to managing oversight from multiple regulatory bodies at the local, state, and federal level, but SEC would be a newcomer in our line of work. Their involvement will be accompanied by a range of new requirements that are common for the financial world, but would be unfamiliar to environmental staff.
Across other EHS fields, we are seeing increased demand for transparency in EHS functions. Overall, this is a positive move, as it brings more attention to EHS issues and develops a better EHS culture within organizations. But this also drives the need for better tools to make EHS information readily available across all levels of the organization.
3. What are the most disruptive technologies available today?
As far as technologies, the ones most likely to have significant impact in the environmental field are ones that don’t require a significant capital investment. Although there are definitely some practical advantages to installing smart monitoring devices and other new technologies, procuring the funding for those purchases is often difficult for environmental professionals. Fortunately, there are still many technologies that have already been implemented successfully in other fields, but only need to be adapted for environmental purposes. Even simple changes like using web-based software in place of spreadsheets can have a huge impact on efficiency. And we haven’t yet seen the full impact of the proliferation of mobile devices on EHS functions. We are still working on new ways to take advantage of mobile devices for data collection, analysis, and communication purposes.
4. What do you think are the biggest innovations of the last 25 years in our field?
We’ve seen a number of innovation milestones in the past 25 years, and while we didn’t invent SaaS, we’ve been largely responsible for adapting it and perfecting it for environmental purposes. One of the major innovations we’ve integrated into our products include online GIS tools where users can easily visualize their environmental data on maps without expensive desktop software. Another one was our fully configurable software platform with built-in form, workflow, and report builders tailored for environmental purposes, which allows anyone to build and deploy environmental software applications that exactly match their needs. There have been many other innovations we’ve incorporated into our software, but these two stand out as the most impactful.
5. Where do you see Environmental and ESG reporting in the future?
More and more, we are seeing all types of reporting being converted into pure data exchanges. Reports that used to include regulatory forms and text interpretations are being replaced with text or XML file submittals. This transition is being driven largely by availability of technology for EHS professionals to generate and read these files, but it is also promoted by regulatory agencies and other stakeholders receiving these reports. Stakeholders have less time to read volumes of interpretive text, and are becoming more skeptical of potential bias in how facts are presented in text. These are driving the need for more pure data exchanges, with increasing emphasis on quantifiable metrics. These types of reports are also more readily compared against regulatory or industry standards. For reporters, lengthy corporate reports with volumes of text and graphics are becoming less common, and the success of an organization’s programs will be increasingly reliant on robust data sets, since ultimately only the data will be reported.
6. What has been the key reason for Locus’ success for the past 25 years?
There are actually a few that immediately come to mind. One reason is the nature of our continually evolving products. By providing our solutions as SaaS, our software adapts with new environmental requirements, and with new technologies. If our software was still the same as it was 25 years ago, it simply wouldn’t be sufficient for today’s requirements. Since our software is updated multiple times each year, it is difficult to notice the incremental changes, but they can be readily seen if you compare today’s software with the original in 1997. And we’re committed to continuing the development of our products as environmental needs change.
The other primary reason for our success is our excellent staff and the environmental expertise we bring to our customers. We simply could not provide the same level of support without our team of environmental engineers, scientists, geologists, chemists, and an array of others. Having that real-world understanding of environmental topics is how we’ve maintained customer relationships for multiple decades. And our software only has value because it is maintained and operated by staff who appreciate the complexity and importance of environmental work.
Mr. Hawthorne has been with Locus since 1999, working on development and implementation of services and solutions in the areas of environmental compliance, remediation, and sustainability. As President, he currently leads the overall product development and operations of the company. As a seasoned environmental and engineering executive, Hawthorne incorporates innovative analytical tools and methods to develop strategies for customers for portfolio analysis, project implementation, and management. His comprehensive knowledge of technical and environmental compliance best practices and laws enable him to create customized, cost-effective and customer-focused solutions for the specialized needs of each customer.
Mr. Hawthorne holds an M.S. in Environmental Engineering from Stanford University and B.S. degrees in Geology and Geological Engineering from Purdue University. He is registered both as a Professional Engineer and Professional Geologist, and is also accredited as Lead Verifier for the Greenhouse Gas Emissions and Low Carbon Fuel Standard programs by the California Air Resources Board.
Want to discuss EHS and ESG solutions with our product specialists? Contact us today!
5 Major Signs That You Need to Replace Your Water Data Management Software
In providing water quality data management software to organizations serving millions of customers a day, our experts have found some common red flags in alternative solutions. Many alternatives to Locus are more prohibitive than helpful, leading to more issues than they should. Your organization deserves to reduce the stress of data entry, regulatory and voluntary reporting, and more. Here are the top 5 signs that your organization is using outdated water quality software:
1. You’re transcribing data more than once or still using paper forms.
In order to ensure the highest level of data quality, you should not be risking human error at multiple levels. Enter your data once, and have it audit-ready, set to go on regulatory and voluntary reports, which are created directly from Locus Software.
2. Product support is not helmed by specialists who support you adequately after implementation.
Support doesn’t end after implementation. What we often hear from our customers when they switch from other providers is that they are delighted with the level of support that Locus brings with our software. Locus is proud to have the expertise and experience to back our software, and if there is anything you need, you can be sure that Locus support can get it done smoothly.
3. Your software has regular or unexpected downtimes.
You need reliability. Your software should be available to you on-demand. Locus is proud to be the only environmental software developer to publicly share our uptime, which is over 99.9%. If you are experiencing downtimes at inconvenient times or for long periods, you should switch.
4. It’s not mobile-enabled.
Field collection is key for most organizations managing water quality data. You should be able to enter that data into your system once, and from anywhere, reducing errors and extra time doing the same work twice (or more!). Also, being able to access historical data at your fingertips can help you solve problems on the fly.
5. It doesn’t provide actionable insights.
Sure, you may have all of your data collected, but what are you doing with it? If your software is not giving you meaningful findings from analyzing your data, then you are always going to be playing catch-up. Having the tools to help your organization look forward is essential in selecting water data management software.
Want to learn more about our Water Data Management Solution? Reach out to our product specialists today!
ESG: Why Uptime of SaaS Vendors Matters
In a Software as a Service (SaaS) delivery model, service uptime is vital for several reasons. Besides the obvious of having access to the service over the internet at any given time and staying connected to it 24/7/365, there are additional reasons why service uptime is essential. One of them is quickly verifying the vendor’s software architecture and how it fits the web.
Locus is committed to achieving and maintaining the trust of our customers. Integral to this mission is providing a robust compliance program that carefully considers data protection matters across our cloud services and service uptime. After security, service uptime and multitenancy at Locus come as a standard and, for the last 25 years, have been the three most essential pillars for delivering our cloud software. Our real-time status monitoring (ran by an independent provider of web monitoring services) provides transparency around service availability and performance for Locus’ ESG and EHS compliance SaaS products. Earlier I discussed the importance of multitenancy in detail. In this article, I will cover the importance of service uptime as one measure to determine if the software vendor is running genuine multitenant software or not.
Service Uptime
If your software vendor cannot share uptime statistics across all customers in real-time, they most likely do not run on a multitenant SaaS platform. One of the benefits of SaaS multitenancy (that is frequently overlooked during the customer software selection process) is that all customers are on the same instance and version of the software at all times. For that reason, there is no versioning of software applications. Did you ever see a version number for Google’s or Amazon’s software? Yet they serve millions of users simultaneously and constantly get upgraded. This is because multitenant software typically provides a rolling upgrade program: incremental and continuous improvements. It is an entirely new architectural approach to software delivery and maintenance model that frees customers from the tyranny of frequent and costly upgrades and upsell from greedy vendors. Companies have to develop applications from the ground up for multitenancy, and the good thing is that they cannot fake it. Let’s take a deeper dive into multitenancy.
An actual multitenant software provider can publish its software uptime across all customers in real-time. Locus, for example, has been publishing its service uptime in real-time across all customers since 2009. Locus’s track record speaks for itself: Locus Platform and EIM have a proven 99.9+ percent uptime record for years. To ensure maximum uptime and continuous availability, Locus provides redundant data protection and the most advanced facilities protection available, along with a complete data recovery plan. This is not possible with single-tenant applications as each customer has its software instance and probably a different version. One or a few customers may be down, others up, but one cannot generally aggregate software uptime in any meaningful way. The fastest way to find if the software vendor offers multitenant SaaS or is faking it is to check if they publish online, in real-time, their applications uptime, usually delivered via an independent third party.
Legacy client-server or single-tenant software cannot qualify for multitenancy, nor can it publish vendor’s uptime across all customers. Let’s take a look at definitions:
Single-Tenant – A single instance of the software and supporting infrastructure serves a single customer. With single-tenancy, each customer has their independent database and instance of the software. Essentially, there is no sharing happening with this option.
Multitenant – Multitenancy means that a single instance of the software and its supporting infrastructure serves multiple customers. Each customer shares the software application and also shares a single database. Each tenant’s data is isolated and remains invisible to other tenants.
A multitenant SaaS provider’s resources are focused on maintaining a single, current (and only) version of the software platform rather than being spread out in an attempt to support multiple software versions for customers. If a provider isn’t using multitenancy, it may be hosting thousands of single-tenant customer implementations. Trying to maintain that is too costly for the vendor, and sooner or later, those costs become the customers’ costs.
A vendor invested in on-premise, hosted, and hybrid models cannot commit to providing all the benefits of an actual SaaS model due to conflicting revenue models. Their resources will be spread thin, supporting multiple software versions rather than driving SaaS innovation. Additionally, suppose the vendor makes most of their revenue selling on-premise software. In that case, it is difficult for them to fully commit to a proper SaaS solution since most of their resources support the on-premise software. In summary, a vendor is either multitenant or not – there is nothing in between. If they have a single application installed on-premise of customer or single-tenant cloud, they do not qualify to be called multitenant SaaS.
Before you engage future vendors for your enterprise ESG reporting or EHS compliance software, assuming you already decided to go with a SaaS solution, ask this simple question:
Can you share your software uptime across ALL your customers in real-time? If the answer is no, pass.
Multitenancy Explained
And if the vendor suddenly introduces a “multitenant” model (after selling an on-premises or single-tenant software version for 10+ years), who in the world would want to migrate to that experimental cloud without putting the contract out to bid to explore a switch to well established and market-tested actual multitenant providers? The first-mover advantage of multitenancy is a considerable advantage for any vendor. Still not convinced? Let me offer a simple analogy to drive home the point as to why service uptime and multitenancy matter: Tesla vs. Edison–War of Currents.
The War of Currents was a series of events surrounding the introduction of competing electric power transmission systems in the late 1880s and early 1890s that pitted companies against one another and involved a debate over the cost and convenience of electricity generation and distribution systems, electrical safety, and a media/propaganda campaign, with the leading players being the direct current (DC) based on the Thomas Edison Electric Light Company and the supporters of alternating current (AC) based on Nikola Tesla’s inventions backed by Westinghouse.
With electricity supplies in their infancy, much depended on choosing the right technology to power homes and businesses across the country. The Edison-led group argued for DC current that required a power generating station every few city blocks (single-tenant model). In contrast, the AC group advocated for a centralized generation with transmission lines that could move electricity great distances with minimal loss (multitenant model).
The lower cost of AC power distribution and fewer generating stations eventually prevailed. Multitenancy is equivalent to AC regarding cost, convenience, and network effect. You can read more about how this analogy relates to SaaS in the book by Nicholas Carr, “Big Switch.” It’s the best read so far about the significance of the shift to multitenant cloud computing. Unfortunately, the ESG/EHS software industry has lagged in adopting multitenancy.
Given these fundamental differences between different modes of delivering software as a service, it is clear that the future lies with the multitenant model.
Whether all customer data is in one or multiple databases is of no consequence to the customer. For those arguing against it, it is like an assertion that companies “do not want to put all their money into the same bank account as their competitors,” when what those companies are doing is putting their money into different accounts at the same bank.
When customers of a financial institution share what does not need to be partitioned—for example, the transactional logic and the database maintenance tools, security, and physical infrastructure and insurance offered by a major financial institution—then they enjoy advantages of security, capacity, consistency, and reliability that would not be affordably deliverable in isolated parallel systems.
Locus has implemented procedures designed to ensure that customer data is processed only as instructed by the customer throughout the entire chain of processing activities by Locus and its subprocessors. Amazon Web Services, Inc. (“AWS”) provides the infrastructure used by Locus to host or process customer data. Locus hosts its SaaS on AWS using a multitenant architecture designed to segregate and restrict customer data access based on business needs. The architecture provides an effective logical data separation for different customers via customer-specific “Organization IDs” and allows customer and user role-based access privileges. The customer interaction with Locus services is operated in an architecture providing logical data separation for different customers via customer-specific accounts. Additional data segregation ensures separate environments for various functions, especially testing and production.
Multitenancy yields a compelling combination of efficiency and capability in enterprise cloud applications and cloud application platforms without sacrificing flexibility or governance.
Want to learn more? Reach out to our product specialists today.
The Horrors of Excel for Data Management
Locus has been preaching on the pitfalls of Excel for a long time. It’s no surprise that one of the worst imaginable errors in Excel that could’ve happened, did. Almost 16,000 COVID-19 cases in England went unreported because Public Health England hit the maximum row count in their version of Excel.
This is not the only example of Excel being misused or being the wrong tool entirely for the job. Excel is not in any way a data management system for complex or vital data. When it comes to sustainability reporting and environmental data management, the evils of the grid are a force to be reckoned with. We have highlighted a few examples that will have you shivering.
Case 1: The Evils of Autofill
Take a look at this harmless-looking chart. It shows monthly electricity consumption for a facility set to report:
| Month | Monthly Electricity Consumption (MWh) |
| January 2019 | 133,500 |
| February 2019 | 122,400 |
| March 2019 | 138,900 |
| April 2019 | 141,600 |
| May 2019 | 141,601 |
| June 2019 | 141,602 |
| July 2019 | 141,603 |
| August 2019 | 141,604 |
| September 2019 | 141,605 |
| October 2019 | 141,606 |
| November 2019 | 141,607 |
| December 2019 | 141,608 |
During review, the auditor notices a distinct trend from April to December, indicating false data overwritten by a stray double-click. Eventually, the auditor required re-entering all invoice data for dozens of facilities to correct the issue. Where the original data went and how autofill went astray remains a mystery.
Case 2: The Phantom File Editor
Imagine using a massive spreadsheet with lots of linked calculations for your annual sustainability report. One of the team engineers works on the file to input more data and get it ready for presentation. But in the final steps, they accidentally delete one of the formulas that sum up the indicators. The annual total looks great for the presentation since you’ve effectively removed a portion of your resource consumption, but afterwards you discover the conclusions were incorrectly calculated. How did that error get introduced? The spreadsheet has no auditing capabilities on the individual values, so you may never know.
Excel supports multiple users editing one document simultaneously, but not well. Multiple records are saved, edits are lost, and vital data vanishes, or at best is very hard to recover. The Track Changes feature is not infallible, and over reliance on it will cause hardship.
Case 3: Date of the Dead
Excel has a frustrating insistence of changing CAS numbers into dates, even if they are something like “7440-09-7″ turning into September 7, 7400. If you’re not explicit in your cell formatting, Excel isn’t happy leaving values as they are.
Case 4: Imposter Numerical Values
You meant to type 1.5, but you typed “1..5” or “.1.5”. Does Excel reject these imposter numbers or let you know of a potential error? No, it’s stored in Text format. This can throw off any averages or sums you may be tracking. This minor identity theft can cause a real headache.
Other Significant Cases:
Other data quality issues with using Excel include, but are not limited to:
- Locations with multiple variations of the same ID/name (e.g., MW-1, MW-01, MW 1, MW1, etc.)
- Use of multiple codes for the same entity (e.g., SW and SURFW for surface water samples)
- Loss of significant figures for numeric data
- Special characters (such as commas) that may cause cells to break unintentionally over rows when moving data into another application
- Bogus dates like “November 31” in columns that do not have date formats applied to them
- Loss of leading zeros associated with cost codes and projects numbers (e.g., “005241”) that have only numbers in them but must be stored as text fields
- The inability to enforce uniqueness, leading to duplicate entries
- Null values in key fields (because entries cannot be marked as required)
- Hidden rows and/or columns that can cause data to be shifted unintentionally or modified erroneously
- Inconsistent use of lab qualifiers— in some cases, these appear concatenated in the same Excel column (e.g., “10U, <5”) while in other cases they appear in separate columns
As you can see, the horrors of Excel are common, and terrifying. Without a proper system of record, auditing features, and the ability for data to vanish into the ephemera, Excel offers little in the way of data security and quality for organizations managing vital environmental and compliance data. Many are learning firsthand the superiority of database management systems over spreadsheets when it comes to managing data. Now is the time to examine the specific shortcomings of your current system and consider your options.
Contact us today to learn how Locus makes complex data management a little less spooky!
6 Ways To Get Data Into Your EHS System
Locus provides multiple methods to populate EHS, ESG, or any environmental data, including the following:
Integrations
Locus provides a full suite of REST API’s, and SDK that can be used to populate data from external data sources. Typical uses include utility data, CEMS, meter data and IoT data.
Surveys
Locus Survey tool enables you to issue survey questionnaires to people outside your organization, and enables them to securely and seamlessly respond directly into the survey form. Typical uses include supplier surveys, audits and customer questionnaires.
Mobile
User input forms can be optimized for input on a phone or tablet, which allows quick uploads of photos and also geotags your data so you can ensure it was collected at the right location.
Excel and Text Files
Locus provides a full suite of Excel upload tools that allow you to import data directly from Excel or CSV files. This option also allows you to work offline and re-sync your data later. Typical uses include laboratory data, periodic monitoring data and data migrations.
Manual Data
Like any system, Locus provides tools for users to directly enter data into the system. These include Locus sophisticated data validation tools which employs machine learning techniques to identify data entries which may be invalid, with visual indications of the expect range or ranges.
Locus can be configured to directly read email input (as text) and place it into the system. Typical uses include instances where external users initiate a conversation, which then may be responded to from within the system, such as an inquiry, issue, or an incident report.
Contact us to learn more
Send us your contact information and a Locus representative will be in touch to discuss your organization’s environmental data management needs and provide an estimate, or set up a free demo of our enterprise environmental software solutions.
Getting Started With ESG Is Less Daunting Than You Think
One of the most frequent questions we get asked when it comes to ESG is, “Where do I begin?”. For many companies, the process of getting started with a new ESG program is the most difficult step. With nearly 1,700 frequently evolving ESG reporting protocols available, it can be daunting just to determine where to begin. This uncertainty associated with ESG reporting can unfortunately paralyze any progress for several organizations. The good news is that ESG doesn’t have to be an ‘all or nothing’ effort. In fact, getting started is a simple and straightforward process.
Regardless of what ESG reporting program you choose (or eventually choose), there are many common elements that can form the basis of your organization’s ESG program. Although social and governance KPIs have been undergoing rapid evolution recently, environmental KPIs have been comparatively stable. Environmental KPIs tend to be quantitative with established calculation methodologies, whereas the definitions and determinations as to what is important regarding societal and governance factors and how to measure them are still being evaluated globally. Considering this, many companies elect to start their ESG reporting program using monitoring and collecting environmental data.
Additionally, almost all reporting programs include the concept of a baseline, or a time period against which future ESG metrics are compared. Developing the baseline requires a good understanding of your organization’s current ESG performance, which of course requires a good set of data. Universal data that is required for any ESG reporting program includes data on greenhouse gas, water quality and consumption, waste, and energy consumption. The bedrock of an ESG program starts with the collection, management, and reporting of these data. This information can also help to inform further decisions for your ESG program, including which framework is most appropriate for your organization.
As part of this effort, you should make sure you are collecting and calculating your ESG metrics with software that supports the required complexity of environmental data. Often the companies who suggest a turnkey solution to ESG reporting are not only lacking in social and governance data, but are woefully underprepared and unequipped to handle environmental data as well. With over 25 years of experience in creating software for environmental reporting, Locus Technologies is equipped to help organizations collect and report ESG data in a way that others aren’t.
Contact Us to Get Started Today
Combat Green Skepticism with Accurate ESG Data
Greenwashing, or the presenting of misinformation to create a sustainable image, is common among organizations. While many consumers may not be aware with the term greenwashing, they are aware of how common it is. In fact, consumers are so aware of this trend that they’re overwhelmingly skeptical of all organizations presenting themselves as sustainable. Four out of every five consumers have expressed skepticism of organizations claiming to be sustainable. So, how does your company express your sincere desire to take steps that are sustainable for the environment? With accurate and transparent data.
With 2/3 of consumers seeking out companies that emphasize sustainable practices, the temptation to greenwash is certainly enticing. Sometimes it comes in the form of making irrelevant claims, like saying a product is free of something that is banned (like CFCs). Other times it comes in the form of half-truths, like saying that a product is sustainably sourced, despite the manufacture of the product being unsustainable or harmful to the environment. Any way you look at it, the demand for sustainable products is so high, as is the temptation to greenwash. The truest, and least disputable way to combat greenwashing is by collecting and reporting your data accurately.
Sustainability is now a broad umbrella term that encompasses not only environmental practices, but social and corporate governance as well, better known as ESG. This broadness reflects a change in consumer attitudes from generation to generation, perceiving more than environmental practices as important while holding environmental practices to the microscope. In fact, over 75% of Gen X consumers say that they have to trust a brand before purchasing from them, and over 85% of Millennial and Gen Z consumers say they same. This trust encompasses everything from the use of organic ingredients to company wellness practices, and is reinforced with buying practices. To do environmental, social, and corporate governance right, organizations have taken a data-first approach.
With Locus Technologies, you can take concrete steps towards achievable ESG goals. By taking a fully-digital approach, your organization can make the transformation by maintaining full visibility of raw sustainability data, calculations, and other factors, and also keeping data easily accessible and traceable. Reports are fully traceable back to the source, and are indisputable, allowing for increased trust from consumers or anyone else who has a stake in this information. Given that 7/10 consumers are willing to pay a premium to sustainable-minded companies who are fully transparent with their efforts, this move can provide a significant return on investment in the short term.
The benefits of data centralization also go beyond combatting greenwashing. A fully-digital and streamlined process will improve your ability to handle the data appropriately, and will ease any auditing and reporting responsibilities moving forward, making the entire process cheaper and faster.
With brand loyalty and purchasing decisions being reliant on sustainable decisions, the move to accurate and transparent data management is key. By implementing Locus Technologies ESG software, your organization can employ cutting-edge solutions to combat greenwashing by promoting your sustainability goals and actions transparently and accurately.
Request an online demo of our ESG solutions
Top 5 Usability Features in Locus EIM
Locus EIM is the leading cloud-based application for managing and reporting environmental data. We have highlighted 5 key usability features that allow users to get the most out of their investment.
Quicker Data Searching with Natural Language Processing
The recent year of lockdowns pushed many daily activities into the virtual world. Work, school, commerce, the arts, and even medicine have moved online and into the cloud. As a result, considerably more resources and information are now available from an internet browser or from an application on a handheld device. To navigate through all this content and make sense of it, you need the ability to quickly search and get results that are most relevant to your needs.
You can think of the web as a big database in the cloud. Traditionally, database searches were done using a precise syntax with a standard set of keywords and rules, and it can be hard for non-specialists to perform such searches without learning programming languages. Instead, you want to search in as natural a matter as possible. For example, if you want to find pizza shops with 15 miles of your house that offer delivery, you don’t want to write some fancy statement like “return pizza_shop_name where (distance to pizza shop from my house < 15 miles) and (offers_delivery is true). You just want to type “what pizza shops within 15 miles of my house offer delivery?” How can this be done?
Enter the search engine. While online search engines appeared as early as 1990, it wasn’t until Yahoo! Search appeared in 1995 that their usage became widespread. Other engines such as Magellan, Lycos, Infoseek, Ask Jeeves, and Excite soon followed, though not all of them survived. In 1998, Google hit the internet, and it is now the most dominant engine in use. Other popular engines today are Bing, Baidu, and DuckDuckGo.
Current search engines compare your search terms to proprietary indexes of web page and their content. Algorithms are used to determine the most relevant parts of the search terms and how the results are ranked on the page. Your search success depends on what search terms you enter (and what terms you don’t enter). For example, it is better to search on ‘pizza nearby delivery’ than ‘what pizza shops that deliver are near my house’, as the first search uses less terms and thus more effectively narrows the results.
Search engines also support the use of symbols (such as hyphens, colons, quote marks) and commands (such as ‘related’, ‘site’, or ‘link’) that support advanced searches for finding exact word matches, excluding certain results, or limiting your search to certain sites. To expand on the pizza example, support you wanted to search for nearby pizza shops, but you don’t want to include Nogud Pizza Joints because they always put pineapple on your pizza. You would need to enter ‘pizza nearby delivery -nogud’. In some ways, with the need to know special syntax, searching is back where it was in the old database days!
Search engines are also a key part of ‘digital personal assistants’, or programs that not only perform searches but also perform simple tasks. An assistant on your phone might call the closest pizza shop so you can place an order, or perhaps even login to your loyalty app and place the order for you. There is a dizzying array of such assistants used within various devices and applications, and they all seem to have soothing names such as Siri, Alexa, Erica, and Bixby. Many of these assistants support voice activation, which just reinforces the need for natural searches. You don’t want to have to say “pizza nearby delivery minus nogud”! You just want to say “call the nearest pizza shop that does delivery, but don’t call Nogud Pizza”.
Search engine and digital personal assistant developers are working towards supporting such “natural” requests by implementing “natural language processing”. Using natural language processing, you can use full sentences with common words instead of having to remember keywords or symbols. It’s like having a conversation as opposed to doing programming. Natural language is more intuitive and can help users with poor search strategies to have more successful searches.
Furthermore, some engines and assistants have artificial intelligence (AI) built in to help guide the user if the search is not clear or if the results need further refinement. What if the closest pizza shop that does delivery is closed? Or what if a slightly farther pizza place is running a two-for-one special on your favorite pizza? The built-in AI could suggest choices to you based on your search parameters combined with your past pizza purchasing history, which would be available based on your phone call or credit charge history.
Searching in Locus EIM
The Locus team recently expanded the functionality of the EIM (Environmental Information Management) search bar to support different types of data searches. If a search term fits several search types, all are returned for the user to review.
- Functionality searches: entering a word that appears in a menu or function name will return any matching menu items and functions. For example, searching for ‘regulatory exports’ returns several menu items for creating, managing, and exporting regulatory datasets.
- Help searches: entering a word or phrase that appears in the EIM help files will return any matching help pages. For example, ‘print a COC’ returns help pages with that exact phrase.
- Data searches: entering a location, parameter, field parameter, or field sample will return any matching data records linked with that entity. For example, searching for the parameter ‘tritium’ returns linked pages showing parameter information and all field sample results for that parameter. Searching for the location ‘MW-1’ returns linked pages showing all field samples, groundwater levels, field measurements, and field sample results at the location.
EIM lets the user perform successful searches through various methods. In all searches, the user does not need to specify if the search term is a menu item, help page, or data entity such as parameter or location. Rather, the search bar determines the most relevant results based on the data currently in EIM. Furthermore, the search bar remembers what users searched for before, and then ranks the results based on that history. If a user always goes to a page of groundwater levels when searching for location ‘MW-1’, then that page will be returned first in the list of results. Also, the EIM search bar supports common synonyms. For example, searches for ‘plot’, ‘chart’, and ‘graph’ all return results for EIM’s charting package.
By implementing the assistance methods described above, Locus is working to make searching as easy as possible. As part of that effort, Locus is working to add natural language processing into EIM searches. The goal is to let users conduct searches such as ‘what wells at my site have benzene exceedances’ or perform tasks such as ‘make a chart of benzene results’ without having to know special commands or query languages.’
How would this be done? Let’s set aside for now the issues of speech recognition – sadly, you won’t be talking to EIM soon! Assume your search query is ‘what is the maximum lead result for well 1A?’
- First, EIM extracts key terms and modifiers (this is called entity recognition). EIM would extract ‘maximum’, ‘lead’, ‘result’, ‘well’, and ‘1A’, while ignoring connecting words such as ‘the’ or ‘for’.
- Then, EIM categorizes these terms. EIM would be ‘trained’ via AI to know ‘lead’ is mostly used in environmental data as a noun for the chemical parameter, and not a verb. ‘Result’ refers to a lab result, and ‘well’ is a standard sampling location type.
- EIM then runs a simple query and gets the maximum lead result for location 1A.
- Finally, EIM puts the answer into a sentence (‘The maximum lead result at location 1A is 300 mg/L on 1/1/2020’) with any other information deemed useful, such as the units and the date.
A similar process could be done for tasks such as ‘make a chart of xylene results’. In this case, however, there is too much ambiguity to proceed, so EIM would need to return queries for additional clarifications to help guide the user to the desired result. Should the chart show all dates, or just a certain date range? How are non-detects handled? Which locations should be shown on the chart? What if the database stores separate results for o-Xylene, m,p-Xylene, plus Xylene (total)? Once all questions were answered, EIM could generate a chart and return it to the user.
Natural language is the key to helping users construct effective searches for data, whether in EIM, on a phone, or in the internet. Locus continues to improve EIM by bringing natural language processing to the EIM search engine.
About the Author—Dr. Todd Pierce, Locus Technologies
Dr. Pierce manages a team of programmers tasked with development and implementation of Locus’ EIM application, which lets users manage their environmental data in the cloud using Software-as-a-Service technology. Dr. Pierce is also directly responsible for research and development of Locus’ GIS (geographic information systems) and visualization tools for mapping analytical and subsurface data. Dr. Pierce earned his GIS Professional (GISP) certification in 2010.
Tag Archive for: Data Quality
Nothing Found
Sorry, no posts matched your criteria