The data you get out of the system is only as good as the data in it. It is not just invalid data that affects what you get out of your database but the completeness and correctness of the data.
Today there is more than one way that invalid, inconsistent or missing data gets into your database.
Internal staff additions and updates are just part of the picture. A growing amount of data is coming from websites that add and/or update data in your database.
Your best line of defense is limiting the chance of invalid, inconsistent or incomplete data at the time of input. It is easier, and less work, to have validation checks on data at the time it is input than have an issue surface later that has to then get resolved or goes unnoticed and lessens the quality of your data.
The first step is to have policies in place outlining your association’s data policy. That may include outlining what data you capture, fields that relate to each other and how to handle situations where required data is not provided.
AM.NET’s standard programming has validation checks to make sure required data is captured such as when you change someone’s status to member that there is a join date on the record. The system though does not validate input set by your internal policy such as when a position description is input that a position code is also selected. In general AM.NET does not do a validation on these two fields because not all societies track data in both fields or if they do they do not require a position code when there is a position description. Custom programming can be added for you association so the system will make a validation like this.
Your website should also have validation checks in its programming to limit invalid, inconsistent or incomplete data from getting into your database. In a way, this can be more important for an association to set policies for because you have no control over the website user who is doing the input like you do on your staff for internal input.
For example, an employee should know that when adding a new firm record that they will be required to select a general business for that firm. If your website allows the addition of a new firm, and does not allow for input, or have validation on a general business field requiring it when a new firm is added, then when that new firm is added to your database, it will later fall on your staff to enter that required data. They may not be able to determine what the general business should be which means even more extra work to contact the firm and get the information before any edit can be made on the new firm record.
On your website, make sure that all required fields first are part of the data being captured and second that those fields are validated to not be blank. Also anywhere that fields relate to each other, such as position code and position description, or general business and specific business, make sure that the website does not allow the user to make inconsistent selections between the two fields. An example of this would be having a general business of public accounting and a specific business of manufacturing.
No process can be 100 percent so even with processes in place to limit bad data from getting into your database there will always be bad data. So checks need to be done on the existing data in the database to get that data as clean as possible. It is less work in the long run if these checks are done frequently. Issues will surface more quickly and there will be less data to fix.
The reports/routines listed below can be used to help in locating bad data in the database. A smaller association may run these reports/routines less frequently than a larger association because they have less data coming into their database, but they should be run as frequently as feasible to surface issues as quickly as possible.
These reports/views can be used to find bad data.
Dues – Reports – Other – Error Report – Run Report Monthly, if errors found Contact CDS
Events – Reports – Financial – Error Report– Run Report Monthly, if errors found Contact CDS
Review – Reports – Financial – Error Report– Run Report Monthly, if errors found Contact CDS
AP – Reports – Other – Error Report– Run Report Monthly, if errors found Contact CDS
Events – Reports – Money on Account – MOA Error Report– Run Report Monthly, if errors found Contact CDS
Firms – Reports – Other – Invalid Codes List – Run Report Monthly
Names – Reports – Other – Invalid Codes List – Run Report Monthly
Events – View - sort or group data to see anomalies in data – use as needed
Names View – sort or group data to see anomalies in data – use as needed
In addition to the Views listed above, saved criteria can be created in selection criteria to help locate bad data.
Delete e-mail addresses with hard bounce back to keep off blacklist
These report/routines can be used to help maintain good data.
Names – Reports – Other – Duplicate Names– Run Report as often as feasible
Firms – Reports – Other – Duplicate Firms– Run Report as often as feasible
Names File Maint – Routines – Delete and Merge Name– Run Routine as needed
Firms – Routines – Special Routines – Merge Two or More Firms– Run Routine as needed
Names – Reports – Mailings – Address Errors – Run Report as needed
Firms – Reports – Mailings – Address Errors– Run Report as needed
Firms – Reports – Other – Unlinked Firms List– Run Report as often as feasible
Events – Routines – Special Routines – Update Registration Counts– Run Routine as needed
Firms – Routines – Special Routines – Update Employee Counts– Run Routine as needed
Utilities – Routines – Supervisor Functions - Email Domain Update– Run Routine as needed
Firms – Routines – Special Routines – Flag Unlinked /Inactive Firms for Deletion– Run Routine as needed
Names – Routines – Special Routines – Flag Inactive Nonmember Records for Deletion– Run Routine as needed
Standardization on entry of addresses
Utilities – Routines – File Maintenance – Zip Code File Maintenance (auto populate chapter, county, city)
Review user defined lists (limit users who can enter new codes)
Utilities – Routines – Supervisor Functions – Delete, Consolidate– Run Routine as needed
These reports/routines do not relate to bad data but should be run on a regular schedule.
Utilities – Routines – General Ledger Journal Entries – Money on Account Balance Write Off – Run Routine as needed
Products – Reports – Other – Small Balances– Run Report Monthly
Products – Routines – Special Routines – Small Balance Adjustments– Run Routine as needed