Introduction and Installation

Available in evaluation mode (no changes are saved) in all versions of Postcode Plus. You can download our evaluation installer here.

If you have purchased Postcode Plus with Refiner, changes can be saved.

Introduction

Refiner cleans address lists or databases containing UK postal addresses against the definitive Royal Mail Address File (PAF). It provides the highest possible levels of accurate automated cleansing and correction and an easy interactive mode.

Launch Refiner from the separate icon in your Postcode Plus group, select Start – Programs – AFD Postcode Plus – Refiner. Alternatively from Postcode Plus you can launch the product by clicking Refiner from the Tools menu.

Backup & Quality Control

Refiner can make substantial changes to your address list, so you should never process a list until a FULL AND SECURE BACKUP is taken. You are strongly advised to work on a copy of your database rather than on live data – you may wish to compare the data after changes with the old, unchanged list.

Even if you write the cleaned data out to separate fields in your database or choose to output to a different database, you should still ensure your data is backed up to ensure you do not lose anything should you set something up incorrectly. A mistake should be easy to spot and correct, but if your data is not backed up then you will have no way of returning to your original data to correct that mistake.

You must employ proper quality control procedures to ensure you are happy with the changes made to your database before finally accepting the results and using them in a mailing.

Technical Support

Making sure that our customers get the best from our products is important to us – so long as you have a current licensed copy of a Postcode product, we’re here to help:

World Wide Web

This major resource provides 24 hour, direct access to all AFD’s technical resources. If you have a question, need program examples or require a skilled Postcode dealer to help with your system, start by visiting our web site at: www.afd.co.uk or email your request to support@afd.co.uk.

HelpDesk

If you don’t yet have Internet access, our HelpDesk on 0333 433 0712 will help ensure that your copy of the program is installed and functioning correctly. The HelpDesk is manned Monday – Friday 9am – 5pm (except holidays). Ensure that you are next to your computer, which is switched on prior to calling.

These services provide free, unlimited support about all aspects of using the postcode product range. Whilst we will endeavour to help with general advice, we cannot assist with questions or problems that are not related to your use of an AFD Product – nor can we resolve hardware or network errors. You should refer these questions to your System Manager.

Data & Data Updates

Address data is constantly changing as a result of property redevelopment, business moves and the increasing trend towards the automation of mail delivery. To ensure that you get the maximum benefit from your investment in AFD Postcode programs, you should ensure that the data you are using is as up-to-date as possible. With Refiner you can update your database with Postcode, town and other address changes that Royal Mail make to ensure your data contains, as far as possible, correct addresses for ensuring fast and correct delivery today.

AFD have invested in state-of-the-art technology to provide you with the best possible address data within our programs. Taking Royal Mail data direct and converting it ready for release to our customers in just three days, means that AFD Data is often weeks and sometimes months more up-to-date than data from other sources.

Four data updates are issued each year – usually in early March, June, September and January – subject to Royal Mail delivery.

Make sure that you place your order so that each optional data update is sent to you as soon as it is prepared – or join the POP scheme so that all costs of Data Licenses, Program and Data Updates are covered in one modest standing order payment and shipped automatically when available.

Opening Your Database

From the Refiner ‘File’ menu, click on ‘Open Database’ . Locate and select the database to be cleaned and a pop-up menu will then allow selection of the specific database table containing the address data to be cleaned.

Supported Database Formats

Refiner can operate with Microsoft Access databases (Access 2 to Access 2002), xBase files (dBase, Foxpro, Clipper, etc), and Paradox tables. It also has native support for delimited (e.g. quoted comma separated) and fixed record length ASCII Text files.

Most other databases can be accessed through Refiner’s ODBC support, providing a DSN for that data source has been set-up on your system.

Opening Non-ODBC Databases

Use the ‘File’ Menu ‘Open Database’ (Ctrl+D) option to open non-ODBC databases. Select the database file you wish to work on and Refiner pops up a menu. From this menu choose whether to work on a straight table of data (an Access database may offer several tables within the database file), or whether to use an SQL query. If you select to use SQL Query, a simple SQL Query is provided for access to the data in the first table. This is a starting point and the Query can be modified to suit your needs.

At the end of the SQL Database Query used to open the database, Refiner automatically adds ‘ORDER BY Postcode’, (where ‘Postcode’ is the name of your postcode field if Refiner can determine it). This causes records to be processed in postcode order which, due to the way caching works, improves Refiner’s performance. To process the database in record order simply remove this part of the query and click ‘Query’ to update. If ‘ORDER BY Postcode is not added, or the field name is incorrect, just add or correct it to enhance performance.

Opening Delimited Text Files

Refiner includes very flexible delimited text file support for working with data in text format. This is particularly useful if your data format is not supported, as you can convert a copy to one which is, so that it can be processed. Most databases can export to Comma Separated Text (sometimes known as CSV or ASCII). As delimited text has it’s limitations, If your database supports exporting fixed format text you may prefer to use that instead (see next section). Refiner can only write back to fixed record length files it is reading data from – so if you process a delimited text file, a new file is created with the file suffix ‘.new’, e.g. processing ‘Address.Txt’ would create ‘Address.New’. It is helpful if the first ‘Record’ in your text file has the names of the fields in each field – like ‘Street’, ‘Town’. Refiner can then display these during attachment. SQL cannot be used to work with Text Files – so queries and sorting should be carried out before processing through Refiner.

When you open a delimited text file you will be presented with the following options to specify the format of that text file:

The delimiter is the character which separates each field in the text file, e.g. a comma is the most common here. The qualifier is the character used to help prevent field cross-over as mentioned previously. If there is no qualifier present you can select None from the drop down list. Address list data is especially susceptible to spurious delimiter characters – like quote marks or commas. House names entered as “The Willows” or address lines like Rose Cottage, High Street, can mean that any data process thinks there are more fields than there really are. This is a problem. It’s best to use Quoted Comma Delimited text which is less sensitive to errors. If there are problems in your data export Refiner protects against the address record cross-over that can occur in these circumstances – and limits damage, usually just to one record . It remains far better if these characters are removed before creation of your processing file. The use first record for field names option specifies that the first record in the text file contains the field names – it is recommended you include these when exporting your data to make identifying and mapping fields much easier.

Opening Fixed Record Length Text Files

Fixed record length text files are a very efficient file format and all Refiner features are supported when using them. If your database is not directly supported, or you want maximum speed and flexibility from Refiner this may be the best format for you. The only format which is as efficient as Fixed Record length text files for Refiner to process is dBase files, Paradox tables come a very close third.

When you open a fixed record length text file you will be presented with the following options to specify the format of that text file:

Refiner will attempt to automatically detect the format of the text file, but if you need to adjust this because there is no record separator so the record length cannot be determined or you need to tweak field positions then you can enter these directly here. You can also review some of the records in the file on the right using the navigation controls above the Cancel button to ensure the format is correctly specified.

Opening ODBC Databases

Use the ‘File’ Menu ‘Open ODBC Connection’ option to open a connection to an ODBC database that you have configured a DSN for on your system. The screen will change to allow you to select a DSN, and type your user identifier (uid) and password (pwd) if required and any additional parameters you may require to connect to that database through ODBC. You can then enter the SQL Query to access the data to be cleaned. Then click the ‘Query’ button to connect to your database.

To improve performance, add ‘ORDER BY Postcode’, (where Postcode is the name of your postcode field) at the end of the SQL Query when opening Microsoft Access or ODBC databases. If the postcode is held in two separate ‘outcode’ and ‘incode’ fields this can be specified by using ‘ORDER BY Outcode,Incode’ etc.

Using SQL

Using SQL to query a database is a powerful, flexible way to control the cleaning process. To use it means learning some simple SQL statements. SQL can be used to restrict cleaning to just parts of the database (perhaps where postcodes are missing), to re-start a cleaning session where it was left-off – or as shown to speed up processing by presenting data in postcode order – which is far more efficient on disk access.

The precise form of these examples will vary with the name of the database and the name of the fields within it, but should be easy to translate:

Select * from [Addresses]

Selects all fields from a table called ‘Addresses’. Every address record will be processed.

Select * from [Addresses] where Postcode is null

As above – but only processes address records with a blank postcode field.

Select * from [Addresses] where Len(Postcode) < 6

As above – but only processes records where the data in the postcode field is too short – e.g. B1 instead of B1 1AA.

Select * from [Addresses] where Postcode is not null order by Postcode

As above – but only processes records with postcode field populated and processes the database in postcode order. (this can significantly improve processing speed!).

For full details of supported SQL statements please refer to your database engine’s own documentation.

Use Progress Bar

The Progress Bar on the Process Tab gives an ongoing visual indication of how much of a database has been cleansed. Refiner counts every record when a database is opened or re-queried so that an accurate Progress Bar can be displayed. This database scan takes time and may be undesirable. The progress bar can be switched on / off from the ‘Progress Bar’ option on the ‘ Options’ menu.

Selecting Input Fields

From the ‘Input’ tab (shortcut key [Alt]+2) select from the ‘Available Fields’ those which contain address data and transfer them to the ‘In-use’ list.

Selecting Fields

All fields containing address elements must be transferred to the ‘Address Matching Fields’ list using the buttons in the centre of this screen. Do not transfer any fields that are used for non address data, as Refiner will attempt to match these to address data and include them in the address. This could result in incorrect results and surplus information included in the addresses written back to your database.

  • Add all fields from the ‘Available Fields’ list to the ‘In-use’ list
  • Add the selected fields in the ‘Available Fields’ list to the ‘In-use’ list
  • Remove the selected fields from the ‘In-use’ list
  • Remove all fields from the ‘In-use’ list

To select an item in either list box simply click on the field name. To select multiple fields click field names while holding down the ‘Ctrl’ key on your keyboard or to select a range of fields hold down the shift key and click the first and last field in the range you wish to select before releasing the Shift key.

We recommend that you test Process a number of records interactively to check that fields are correctly mapped. If the correct Input Fields have not been selected simply return to this screen to make corrections.

Field Order

Fields are displayed in the ‘Available Fields’ list in the order they are held in your database. Normally this will mean that they are in the correct order for the address. However if you add an additional field that you earlier missed, or the fields are not in the correct order, use the up and down arrow’s on the right of the ‘In-use’ list to correct this.

For address cleaning to be effective it is important to ensure the ‘In-use’ list is in the correct order of address elements, i.e. the order you would write them on an envelope.

Name Fields

Refiner does not contain name data. However, if name fields are transferred to the ‘In-use’ list it’s possible to use the name as part of the de-duplication process or to process the name data to combine name elements into a single field or split a field into it’s constituent parts.

For each name field in the ‘In-use’ list you must select the field and check the ‘Name Field’ box to the right of the ‘In-use’ list and select from the list of options below the type of name field that field is. This will prevent Refiner incorrectly including these as part of the address. The only fields that Refiner will specifically recognise as a name if this has not been indicated are those starting with a commonly used title (e.g. Mr, Miss, Mrs, Ms, Prof, Doctor etc.).

Organisation Field

This option is for use when cleaning databases where there is a separate ‘organisation’ field which has been consistently used.

PAF contains around 1.5 million organisation names. Transferring the organisation field from the ‘Available Fields’ list to the ‘In-use’ list will enable a higher match rate during cleansing, especially for organisations with a PO Box address.

Some addresses match to a PAF record but contain an ‘extra’ field above the address. In these cases Refiner must place that information in either the property or organisation field. The process for deciding this is usually accurate. However, by selecting the ‘organisation’ check box, the organisation field is fixed and if there is any uncertainty then this field is automatically retained as it appears and is not transferred to the property field.

DX Fields

This option applies only to DX Customers. The DX enabled version of Refiner includes the option to mark Input fields as containing a DX Number and Exchange. These are for when your database contains an existing DX Number field, which should be mapped in and marked appropriately.

Should you have no seperate DX Number field but some of your Input addresses are DX addresses there is no need to mark anything – Refiner will extract the DX details from them automatically.

Specifying Output

The Output tab (shortcut key [Alt]+3) is used to specify how Refiner writes back address data to your database. For databases where there is a separate field for each address element, simply drag the field name from the list on the right (‘Postcode Plus Fields’) to the appropriate yellow box on the left. Refiner is very flexible and supports writing back to address databases in highly complex or unusual formats.

Mapping Fields

From the list beneath ‘Postcode Plus Fields’ caption (right of screen), choose and map Postcode Plus fields to your database. Click the left mouse button down on a field and drag it over to the yellow box corresponding to the field you wish to map to… then release the mouse button. Use the scroll bar to move up and down the ‘Postcode Plus Fields’ list to find further fields for mapping.

More than one ‘Postcode Plus’ field can be mapped to the same (yellow) destination field in your database. For example, if there is no output field for locality, then the locality will need to be dragged to either the street or town field of your database. This is called concatenation. By default, Refiner will separate concatenated fields with a comma and place them in logical order. To change this order or the field concatenator, double click the relevant yellow box – see Changing Field Options below for more details.

Please note that for a complete mailing address all fields from Organisation through to Postcode (county is optional) must be mapped back to your database.

Delivery Point Suffix (DPS). A DPS code is only available for a letterbox level address and not for a street level address. So, where DPS Codes are mapped from the ‘Postcode Plus Fields’ list to a database, Refiner will map back a default DPS of 9Z for addresses which can only be matched to street level. This will help reach maximum Mailsort discount. To switch on /off the writing back of a default DPS toggle the ‘Write back Default DPS’ item on the Options menu.

Selecting the ‘Special’ radio button (beneath ‘Postcode Plus Fields’ caption) will change the list to display raw PAF fields. This enables, for example, building number to be separated from street name and the outcode / incode portions of the Postcode to be separated. (the part before and after the space respectively) . You can mix special and normal fields in your mappings. See Special Fields below for more details.

Only map back Name fields (Name, Title, First Name, Middle Name and Surname) if you have selected them as Input fields for processing these (such as splitting a name into it’s constituent parts). Name data is not included as part of the Refiner. So, if name fields exist on your database and have NOT been mapped to ‘In-use’ – then mapping Postcode Plus Fields to your name fields on the Output tab will overwrite them with blank data.

Where a name field/s are transferred to the ‘In-use’ list Refiner can be set to split or combine fields. Simply map from the Postcode Plus Fields list to the appropriate output field/s (using drag and drop)

Special Fields

Standard Fields take care of the many anomalies and inconsistencies of raw PAF data. We recommend that you use these fields for mapping wherever possible.

Selecting ‘Special’ from the drop down list underneath the ‘Postcode Plus Fields’ caption will change the list of fields to show standard PAF (unformatted) address fields as well as separate Outcode and Incode fields. Select ‘Standard’ from the drop down list to switch back to the standard fields and map different fields from each list.

The fields provided in this list are as follows:

  • Department
  • Organisation Name
  • Sub Building
  • Building
  • Number
  • Dependant Thoroughfare
  • Thoroughfare
  • Dbl-Dep Locality
  • Dependant Locality
  • Abbrv. Postal County
  • Optional County
  • Abbrv. Optional County
  • Traditional County
  • Administrative County
  • Outcode
  • Incode
  • Barcode
  • <Transparent>
  • <Blank>

When using Special Fields map all or none of the fields from Building Number through to Dependant Locality and do not mix and match with the Property, Street and Locality fields from the ‘Standard’ list. To mix these fields will cause some addresses to be trashed. This is because the standard fields not only concatenate many numbers with the street but also sometimes move the property or concatenate to the locality field as appropriate according to the PAF formatting rules.

Address elements not provided on the Special Fields list are excluded because they are identical to those on the Standard Fields list, e.g. the Town field is never reformatted. If you need to map the building number separately but only have a single property field you can drag both the sub building and building fields to the same field in your database and they will be concatenated together as appropriate.

You should note that the Organisation field in this list is different to that on the Standard fields list. In the standard fields list Organisation appears concatenated with the Department (when present), here the Organisation field is separated out from the Department. Similarly the Street field on the standard list is concatenated with the dependant street (where present), here they are separated.

For example, if your database has standard address fields from Organisation through to Postcode, but has a separate Number field you would make the following mappings:

Database FieldField TypeField Name to Map
OrganisationStandardOrganisation
PropertySpecialSub Building, Building
NumberSpecialBuilding Number
StreetSpecialDependant Thoroughfare, Thoroughfare
LocalitySpecialDbl-Dep. Locality, Dependant Locality
TownStandardTown
PostcodeStandardPostcode

Note: You concatenate fields together, e.g. ‘Dependant Thoroughfare’ and ‘Thoroughfare’ together as above by dragging the first field (Dependant Throughfare) and then dragging the next one (Throughfare) to the same field.

The field does not affect what is written back to your database. It is simply used if you wish to view a field unaltered in the suggested record when carrying out any manual cleaning through Refiner. As with any field with nothing mapped Refiner will not alter or write back any field that is mapped too. You only need to map this field if you specifically need to view non-address fields to assist with manual cleaning.

DX Fields

For DX customers, the DX Enabled version of Refiner includes a DX Field Output group. You can use this to write out DX data to your database. By mapping out the DX Number and DX Exchange you will have these supplied wherever they can be successfully matched. Existing DX data in your database will be retained, but the DX Status field will help in examining those that are or may be incorrect. See the DX Status Codes section for details of what each value of the DX Status field means.

Changing Field Options

To remove all field mappings from a field in your database simply drag those fields back to the field list box. You can also move mappings from one field to another by dragging them between fields.

To remove a field mapped to one in your database, to rearrange the order, or to change the field concatenator, simply double click the yellow box containing the field mapping to display the following dialog:

To remove a Postcode Plus field simply select it in the list and click the ‘Remove’ button. To re-arrange the order in which more than one Postcode Plus field is written back to your single database field, use the Up and Down arrows above and below the ‘Remove’ button. The ‘Field Separator’ option to the right can be used to select how fields are concatenated together when written back to a single field in your database. The option is not needed and not available for use to pre-cede the first of the concatenated fields. Refiner will only use the Field Separator where there is data to separate. The ‘Use for all’ option applies the chosen Field Separator to all fields in the list. To specify the different Separators for individual fields – uncheck ‘Use for all’

Line Squeeze

We strongly recommend that your database is structured with fields for a minimum of each of the following address elements: Organisation, Property, Street, Locality, Town, County, Postcode and that these fields are used consistently. (obviously you may want to add others such as Department or DPS code etc).

However, sometimes an adequate number of fields are not available and / or these are restricted in length. In these cases it’s useful to use ‘line squeeze’ Line Squeeze means for example, that if no locality data exists in an address record Town would be moved up to occupy the blank locality field and County would be moved up to occupy the blank town field. Refiner allows you to “line squeeze” fields in your database and to select which fields to “line squeeze”.

To Line Squeeze fields together simply click the check box that follows that field name. You will see the fields join as you check more than one consecutive field. It is not possible to line squeeze non-consecutive fields.

Cleaning Settings

The Settings Tab (Shortcut Key [Alt]+4) is where you specify the options for how Refiner cleans your database.

Note that you can reset Refiner cleaning settings to the defaults by selecting the “Reset To Default Settings” option on the File Menu.

Cleaning Mode

Here you can select the mode that Refiner operates in for cleaning your database. The options available are as follows:

Cleaning ModeDescription
DefaultAttempts all matching methods to match the addresses in your database.
Verify Postcode OnlyAttempts to match addresses only by looking up the postcode and comparing it to the given street address. Although very fast (it avoids searching) where the Postcode is incorrect addresses will not be matched.
Full Match OnlyNormally if Refiner finds a property that is not in PAF but the address can be verified to street level it will return a Street Match and retain all original property information. ‘Full Match Only’ will not return data unless it can be fully matched to an address in PAF. i.e. a full Delivery Point match.
Attach ModeIn attach mode the postcode is looked up and data is “attached” based on the postcode that is supplied. For more details see the Attach Mode section of this manual.

Field Options

The ‘Target Fields’ list on the left of this screen displays the fields in your database that will receive data output from Refiner. You have mapped Refiner fields to these fields and they will be written back to. For each ‘Target Field’ you can select from the list of conditioning options. This is particularly useful to tidy up data that can not be PAF matched. Using conditioning options does not affect Refiner’s ability to clean data – it simply formats output data.

The options available for each field are as follows:

Conditioning OptionDescription
Format Name…This allows you to specify how you wish names in the source data to be formatted. This is only applicable to Name fields.
Initial From First NameExtracts an Initial from the First name (uses the first character). Only applicable for First Name fields
Expand AbbreviationsWhere address data cannot be matched, expands out abbreviations in the address line. e.g. Rd to Road and Ave to Avenue etc
Remove Directives (e.g.. c/o, via)Removes un-necessary directives from the address even when that data cannot be matched. (Address data matched to Royal Mail will be returned in it’s postally correct form).
Remove Unwanted LinesRemoves un-needed address lines e.g. Wales or Scotland
Cleanup Address LineGeneral cleanup – removing general garbage from the field e.g. spurious commas and characters
Remove Repeat Address LinesRemoves duplicated address lines
CapitaliseAttempts to correctly capitalise the address field – e.g. “12 High Street”
Upper CaseForces the address field into all Upper Case – e.g. “12 HIGH STREET”
Maximum Line LengthSpecifies a maximum length for the field and Refiner will sensibly abbreviate to attempt to meet. Where not met, Refiner truncates to fit.

Process Options

For most uses Refiner will clean up your data correctly with it’s default settings. However, if you have advanced requirements the “Advanced Settings” dialog includes additional settings for Refiner which should be used with care.

These are documented in the Advanced Settings section of this manual.

De-Duplication

Refiner includes a de-duplication facility to enable duplicate records to be identified and / or removed from your database. Various criteria can be set to tailor de-duplication to your needs. You must map the ‘Duplication Group’ field from the ‘Postcode Plus Field’ List to a field in your database. Refiner will write a duplication key value to this during the cleaning process. At the end of the process Refiner will mark duplicates with the same duplication group number. All non-duplicating records will have a duplication group number of zero. Records sharing the same group number can later be checked and removed as necessary. Please note that you must move forwards through your database for this feature to work correctly.

To turn on de-duplication check the ‘enable’ box.

De-Duplication is not available for text files.

The options provided for the de-duplication process are as follows:

OptionDescription
Use Spelt LikeSelecting this option takes account of minor typos and spelling errors when comparing records for de-duplication. This is useful when matching with property option selected on a database where many records are only matched to street level.
Use Sounds LikeSelecting this option takes account of similar sounding data when comparing records for de-duplication. This can help with street level matches and organisation comparison but may lead to some mismatches.
Full DPS Matches OnlySelecting this option will mean that only records that have been matched by Refiner to full DPS level (i.e. an exact PAF match) will be compared and marked for de-duplication.

The de-duplication process will always use the postcode. The Duplicate Postcode option can be combined with one or more of the additional fields displayed:

Field OptionDescription
NameSelecting the Name option will identify duplicate names on the same postcode… along with other selected options. A name field/s must be mapped for this feature to work. This selection provides a set of name options to chose from: Full Name, Surname, Surname and First name and Surname and Initial. If no name is supplied, other records with no name at that postcode will be taken as duplicates.
OrganisationSelecting the Organisation option will identify duplicate Organisations on the same postcode. (records with no organisation are taken to be duplicate if all other options match).
PropertySelecting the Property option will identify duplicates where the property is also a duplicate. It is recommended because it prevents identifying a record as a duplicate simply because another record on the same postcode happens to have the same details.
DPSSelecting the DPS option will identify duplicate records only if they share the same DPS i.e only those records that would go through the same letter box
Custom FieldSelecting the Custom Field option allows you to choose any field from your database to use in the de-duplication process.

Duplication results are available from the Results tab from which a report can be viewed, printed and saved. Although duplicates may be deleted directly using Refiner, AFD recommend that you use the duplication group numbers to group and review duplicate records in your own database.

Saving Settings

Refiner automatically saves database settings, attachment information, and Conditioning options on exit. To maintain multiple configurations you can use ‘Open Session’ and ‘Save As Session’ from the File menu to save and retrieve settings. Refiner opens the last used session on start-up.

Processing Records

The Process tab (keyboard shortcut [Alt]+5) is used once you have completed setting up Refiner to clean your database records. You can at any time switch to the ‘Settings’ tab to change your settings.

Viewing records

Moving through the database in interactive mode (default) Refiner displays the original record (as read from your database) on the left of the screen and the suggested record on the right. Fields highlighted in red contain data that has not been matched and therefore has been copied over from the original record.

You can edit the displayed original record to help matching and modify the suggested address. Changes will only be written back to your database if you have selected ‘Store Changes’ from the Options menu. It is useful initially to review the Suggested Records created by Refiner, and use this to adjust your Settings accordingly before committing to store changes.

During processing, Refiner will use a variety of methods to assemble a suggested address from the Original Record. It will use the postcode if present, but will perform powerful quality checking to help ensure the postcode is sensible – and will use other parts of the address to generate a postcode where problems are found. A brief description of how the address was validated is shown on the Refiner status bar (beneath the tabs). If an address cannot be matched in it’s entirety, Refiner will attempt to match to street level and if ambiguous will list possible options.

Fast Find Short Cut

A right mouse click on a Suggested Record field name initiates a Postcode Plus Fast Find search of the data in that field. The operator can accept a search result by highlighting the record in the Postcode Plus list box and clicking the ‘Borrow’ button on the Refiner Process screen. Remember too, that you can edit the Suggested Record field data manually before using the Fast Find short-cut.

Database Movement

To accept changes and move to the next or previous record (previous record not possible with Text databases), simply click the video-type movement buttons. No changes will be written to your file unless ‘Store Changes’ is checked on the Options menu.

Automatic Processing

Selecting ‘Auto Process’ from the Options menu instructs Refiner to verify and correct addresses without user intervention. By selecting ‘Halt On No Match’ from the Options menu, the automatic cleansing process will halt when an address cannot be automatically verified. Using this facility allows a one-pass process in which addresses are automatically verified wherever possible – and the operator deals with the remainder. This option can be selected or deselected at any time.

Before selecting ‘Store Changes’ from the ‘Options’ menu, an initial review of the Suggested Records created by Refiner and any necessary adjustment to settings is recommended. An attempt to start Automatic Processing without selecting ‘Store Changes will result in a warning.

To protect you from serious mistakes in updating, Refiner will only auto-update records where it is reasonably confident that the result is sensible – otherwise records will not be changed. For example where the result is ambiguous it will not be written back to your database. By storing the Result Code to your database, these records can be extracted later for interactive processing.

Ambiguous Matches

Where refiner finds more than one possible address match, a list of possibilities will be presented. In the example below there are 4 postcodes at an identical property address. In automatic mode these suggested possibilities will not be written back to the database. However the relevant result code (-2 or -4) will be ready for any manual checking later. All fields mapped back to your database are shown so you can see what will be written back to your database and modify it if required. If you are uncertain of the address yourself we strongly recommend you select to keep the Original Record – the top entry in the list.

To retain the original record from the ‘Ambiguous Match’ dialog press Cancel or use the ‘Esc’ key or select and Accept ‘(Original Record)’ from the list box. To accept and store another record, select it then press the Accept button, or press the return key. The ‘Accept and Continue’ button accepts the record (or Original Record if that is what is selected) and continues a halted Auto Process or moves to the next record in manual processing.

Ambiguous Postcodes are indicated by a result code of -2. Ambiguous address records are indicated by a result code of -4. Result codes and brief description appear in the Refiner tip bar.

Suggested Matches

An address may contain property and street level information that is unique in the country. However if the locality, town and postcode information do not match Refiner will not validate it. Instead it will present a suggested match for the address. This is displayed in interactive mode in the same way as an ambiguous match (shown above) but there will only be two items in the list – the original record and the suggested record. Operator common sense is used to make a sensible choice.

Reviewing Non-Matched Records

Selecting ‘Review Non-matched Records’ from the Options menu enables an interactive review of all records that could not be automatically cleaned. This option skips records that have already been automatically matched. When you select this option you will be presented with the following options to determine which type of records you wish to review:

The ‘Jump To Record…’ option, also on the Options menu, allows you to jump to the nth record in the order in which data was presented to refiner.

Result Codes

The result codes show to what level and how addresses have been matched:

Please note that result codes 202-205 and 302 are only applicable when the ‘Assume Postcode Correct’ option is used from the ‘Options menu’. This option assumes the postcode given in the address is correct if the address cannot be verified any other way. This will result in a match on the property, postcode only or property and street information only unless other data suggests such a match would not be correct. Records with these result codes in this mode should be treated with caution as they are not completely reliable – that option is intended for databases sparse in data when a most likely address is better than no match at all.

  • 100 – Address verified from Postcode and matches a record in PAF (shown in Postcode Plus) identically.
  • 200 – Address verified from the Postcode and matches a record in PAF (shown in Postcode Plus) with some correction.
  • 201 – Address verified from a postcode which was substituted due to a Royal Mail recoding and now matches a record in PAF (shown in Postcode Plus).
  • 202 – Postcode, Property Match Assuming The Postcode Is Correct
  • 203 – Postcode, Property Match with Changed Postcode Assuming The Original Postcode Is Correct
  • 204 – Postcode Match with Non-Matched Property Added In
  • 205 – Postcode Match with Changed Postcode and Non-Matched Property Added In
  • 300 – Address verified to PAF (shown in Postcode Plus) with some correction, looking wider than just the specified Postcode.
  • 301 – Address verified to PAF (shown in Postcode Plus). Multiple PAF Organisations were listed as this address so the PAF Organisation has not been returned.
  • 302 – Limited DPS Match (Address verified to PAF but Locality or town information not present)
  • 400 – Address verified to Street Level, i.e. the property was not on PAF, but a unique match to the street was identified on a single postcode.
  • 700 – User Modified – You have modified this record yourself to correct it following Refiner’s match attempt.
  • 702 – User Selected from a list of records due to a match with an ambiguous postcode
  • 703 – User Selected Refiner’s Suggested Record (no match was made because there was insufficient data to reliably verify the address).
  • 704 – User Selected from a list of records due to multiple possible matches for the address.
  • -1 – No Match Found – Refiner has been unable to match this record.
    • -2 – Ambiguous Postcode Match – Refiner has matched this record to Street Level but cannot determine which is the correct Postcode and so has presented each of the possibilities.
  • -3 – Suggested Record Match – Refiner has found a unique possible match for this record but there is not enough address data to reliably match it.
  • -4 – Ambiguous Match. Refiner has given several possibilities that this address could match to.
  • -5 – This address was detected as being an International Address and therefore cannot be cleaned as data is only present for cleaning UK, Channel Isles and Isle of Man addresses.
  • -6 – No record data was supplied. Refiner cannot clean this address as no address data was given.
  • -7 – Data Initialisation Error. Check that Refiner is correctly registered by running the AFD Welcome Program.

DX Status Codes

For DX customers with a DX Enabled version of Refiner, the DX Status Field helps identify how reliable DX codes written out are and where existing codes have been retained if they need to be manually examined or corrected:

  • 100 – Confirmed DX match given in the source data matches this organisation at this address.
  • 200 – Identified the DX Number and Exchange for this organisation at this address.
  • 400 – Existing DX number retained and exists but not verified as being correct.
  • -1 – Supplied DX Number does not exist, so is likely to be incorrect unless it is Ex-Directory. -6 – No DX number found or supplied to match this organisation at this address.

Performance

Address list problems vary widely, as do the speeds of the computer systems used to carry out your cleaning operations. It is impossible, therefore to predict cleaning performance. The practice database is designed to have difficult addresses – which take much longer to look up than the average ones, so it would give a very pessimistic indication of actual throughput. Factors affecting performance include:

Hardware

List cleaning is very disk-intensive. Choose equipment with a fast hard disk drive, and install all files (including Postcode Plus data files) on the local hard disk.

Operating Environment

Avoid placing additional load during the cleaning process by running other programs – or acting as a server on a network.

Optimising

There are several steps you can take to make processing even faster for larger lists:

  1. Disk Optimisation – make sure that your disk drive is not badly fragmented – address list cleaning is more sensitive than most to this type of problem.
  2. Record Order – if you are able to sort your data record order so that processing takes place in Postcode order – even if your postcode records are incomplete, you will see significant speed improvements.
  3. Text Files – Processing of Text data is significantly faster than processing the proprietary file types like Access. If speed is very important, it may be quicker to export your data, process it then re-import it into your database program. Text files and dBase files are the most efficient (fastest) formats to use in Refiner, with Paradox coming a close third. If you choose to use delimited text files however you will lose some advanced functionality including Refiners ability to conduct SQL queries and the ability to use de-duplication so we would recommend fixed record length formats are used were possible.

Results

The Results tab (keyboard shortcut [Alt]+6) shows statistics for the cleaning process and also allows you to save and print the report. When de-duplication is being used, the results can also be viewed here.

How Statistics are generated

Each record passed over interactively or automatically by Refiner is added to the totals. Moving back to re-process records will make the results inaccurate as records will be counted twice. For an accurate record count it is important to sequentially process records. The statistics shown are reset when you move to the start of your database or press the Reset button on the Results tab.

The Statistics Explained

Each of the statistics shown on the Results tab show the record count in the first box followed by the percentage of the total records.

StatisticMeaningCorresponding Result Code
Complete PAF MatchRecords matched identically to a record on PAF (The Royal Mail Postcode Address File)100
Postcode MatchRecords matched to PAF from the Postcode.200, 202, 204
… of which ChangedNumber of records that were Postcode matched after the Postcode was changed due to a Royal Mail Postcode recoding.201, 203, 205
Full DPS MatchNumber of records that have been matched to a unique delivery point on PAF after ignoring the postcode300, 301, 302
DPS MatchedThe total number of records that have been matched directly to a record in PAF to Delivery Point Level and therefore can be allocated a DPS (Delivery Point Suffix).Sum of 100’s, 200’s and 300’s
Non-DPS MatchThe total number of records matched to PAF, but not to a specific delivery point.400
Ambiguous PostcodeRecords which could be matched to street level but there is more than one possible Postcode for the address and the correct one could not be determined. No other address details vary.-2
Ambiguous MatchRecords which could not be matched to a unique address. Several possibilities have been given and one or more or none may be correct. The original record will be kept unless you select the correct address yourself.-4
Total AmbiguousThe total number of records for which Refiner has been unable to determine the correct address to match to, but has given a set of possibilities.Sum of -2 and -4
User CleaningThe number of records the user has manually cleaned or modified during the cleaning process700, 702, 703, 704
No Match FoundThe number of records that could not be matched (this includes those where Refiner has suggested a unique match but cannot reliably make that match)-1, -3
Total Non-MatchedThe total number of records that have not been cleaned in the database.Sum of -1 and 700’s
Non UK MatchedThe number of records that could not be matched because they are obviously outside the UK, Isle of Man or Channel Islands (can only be determined if a recognised country is included in the address).-5
No Address DataThe number of records that contain no address data to be cleaned-6
Excluded RecordsThe total number of records that could not be cleaned because they did not include a UK address.Sum of -5 and -6
Records ProcessedTotal number of records processed in the database. 

Generating Reports

The statistics displayed on screen can be saved or printed as a report for reference outside of Refiner or as a permanent record. Pressing the Save button will prompt you for a filename. These are saved as CSV files (comma separated text) for easy manipulation and formatting in your own application or a package of your choice, such as Microsoft Excel. The first row gives the column headings and each subsequent row contains the statistics as shown on the screen. An example of this is:

"Statistic","Number of Records","Percentage of Records"
"Complete PAF Match","278","65.7%"
"Postcode Match","49","11.6%"
" ... of which Changed","0","0.0%"
"Full DPS Match","1","0.2%"
"DPS Matched","22.5%"
"","",""
"Non-DPS Match","0","0.0%"
"","",""
"Total Matches","22.5%"
"","",""
"Ambiguous Postcode","0","0.0%"
"Ambiguous Match","0","0.0%"
"Total Ambiguous","0","0.0%"
"","",""
"User Cleaned","0","0.0%"
"No Match Found","87","20.6%"
"Total Non-Matched","87","22.5%"
"","",""
"International Address","2","0.5%"
"No Address Data","6","1.4%"
"Total Excluded Address","87","22.5%"
"","",""
"Duplicate Records","0","0.0%"
"Duplication Groups","0",""
"","",""
"Records Processed","423","100.0%"

Pressing the Print button will cause Refiner to print a formatted copy of the cleaning statistics. Please note the report is intended to be printed on A4 paper and reports on other paper sizes cannot be guaranteed to be satisfactory. You can select the Printer to use for the reports by selecting the ‘Printer Setup’ option from the File menu. Of course if you want more control over the style of report you print or you need it produced in a certain way saving to a CSV file instead will allow you to generate a report in the way you require.

Duplication

When the De-Duplication process is used the results screen will show how many records are duplicates (Duplication Hits) and the number of Duplication Group (i.e. the number of sets of matching records). These figure are based on the criteria selected for de-duplication.

Pressing the View button will display a report showing each record that may be a duplicate of one or more other records. The example below shows the results of duplication based on postcodes:

To save a copy of this report as a CSV file, click Save. To print a copy of this report, click Print. Select the Printer to use for the reports by selecting the ‘Printer Setup’ option from the File menu on the main Refiner screen. Saving or printing a report can assist the process of removing unwanted duplicate records. The Group Numbers have been written to your database in the field you mapped as the Duplication Group field when you specified field mappings. This means duplicates may be processed and checked directly from your database management system.

Refiner also provides for duplicate record deletion. If your duplication criteria was sufficiently specified to only flag up genuine duplicates (usually means adding at least one other field on-top of the postcode) then you can delete all duplicate records shown (only the first record in each group is retained) by selecting the ‘Delete Duplicates’ button on this form. Before using this process, check the entries shown on this form are correctly identifying duplicates only. If necessary, refine the duplication criteria and run the process again. For more advanced processing of duplicates or selection of records for deletion either remove records manually using your database system or use your own process based on the Duplication Group field to remove records.

Censation

The Censation tab (keyboard shortcut [Alt]+7) is used to display the Censation Report based on the records processed by Refiner.

How Statistics are generated

Each record passed over interactively or automatically by Refiner has it’s Censation Code checked to update the Censation report on this tab. This happens even if you have not opted to write the Censation data back to your database. Moving back to re-process records will make the results inaccurate as records will be counted twice. For an accurate record count it is important to sequentially process records. The statistics shown are reset when you move to the start of your database or press the Reset button on the Results or Censation tab.

The Statistics Explained

The Censation report can help gain insight into the type of categories those on your database fit into. Such as do they tend to be wealthy or struggling, do they tend to be young singles or families, etc. This could help you determine which type of customers it might be most effective to target or help you analyse those statistically most likely to purchase certain products and target them more effectively

Each of the statistics shown on the Censation tab show the record count in the first box followed by the percentage of the total records.

StatisticMeaning
Censation Code RankThis lists the top 12 Censation codes in your database and shows how many records are found in each one. It also shows how many records do not have a Censation code assigned (normally because they were not fully matched) or have another Censation code (not in the top 12). A Censation code is made up of the affluence indicator (first letter), life stage indicator (second letter) and the final two digits point to additional information relating to the code. More details about these codes can be found in the Postcode Plus manual Censation section.
Affluence CategoriesThis shows the number and percentage of records in your database that fall into each of the Affluence Categories (Wealthy through to Struggling). Again more information can be found in the main Postcode Plus manual Censation section.
Life StageThis shows the number and percentage of records in your database that fall into each of the Life Stage Categories. You can select to view the Census Generated codes (those from Young Singles through to Empty Nests and Seniors that are generated from the census data), or the others that are generated from other data sources for area’s not covered by the census. Again more information can be found in the main Postcode Plus manual Censation section.

Generating Reports

The statistics displayed on screen can be saved or printed as a report for reference outside of Refiner or as a permanent record. When saving or printing the Censation report all Censation codes are ranked (not just the top 12) and all Life Stage categories are shown. Pressing the Save button will prompt you for a filename. These are saved as CSV files (comma separated text) for easy manipulation and formatting in your own application or a package of your choice, such as Microsoft Excel. The first row gives the column headings and each subsequent row contains the statistics as shown on the screen. An example of this is:

"Censation Code Rank","","",""
"Position","Code","Number of Records","Percentage of Records"
"1","NL61",6,"27.3%"
"2","EV46",2,"9.1%"
"3","DW33",2,"9.1%"
"4","AX05",1,"4.5%"
"5","EW44",1,"4.5%"
"6","BY21",1,"4.5%"
"7","BY19",1,"4.5%"
"8","BW18",1,"4.5%"
"","None",7,"32.0%"
"","","",""
"Affluence Categories","","",""
"Code","Description","Number of Records","Percentage of Records"
"A","Wealthy",1,"4.5%"
"B","Prosperous",3,"13.6%"
"C","Comfortable",0,"0.0%"
"D","Striving",2,"9.1%"
"E","Struggling",3,"13.6%"
"N","Not Classified",6,"27.2%"
"","","",""
"Life Stage Categories","","",""
"Code","Description","Number of Records","Percentage of Records"
"V","Young Singles",2,"9.1%"
"W","Young Families",4,"18.2%"
"X","Older Families",1,"4.5%"
"Y","Empty Nests and Seniors",2,"9.1%"
"L","Many organisations mixed with low, transient population",6,"27.3%"
"M","Residential development with a high incoming population",0,"0.0%"
"N","Stable turnover of a high population mixed with a few organisations",0,"0.0%"
"O","Stable turnover of population mixed with commercial development",0,"0.0%"
"P","Established high residential population",0,"0.0%"
"","","",""
"Total Records","","22",""

Pressing the Print button will cause Refiner to print a formatted copy of the censation statistics. Please note the report is intended to be printed on A4 paper and reports on other paper sizes cannot be guaranteed to be satisfactory. You can select the Printer to use for the reports by selecting the ‘Printer Setup’ option from the File menu. Of course if you want more control over the style of report you print or you need it produced in a certain way saving to a CSV file instead will allow you to generate a report in the way you require.

For more detailed information about Censation and how it works, what it’s useful for, and what the codes mean please refer to the Censatio section of the main Postcode Plus manual.

Additional Censation Information is also included in appendices G, H, I and J of that Manual:

Attach Mode

Refiner also includes a special Attach Mode. This is activated by selecting the ‘Attach Mode’ option on the settings tab.

In Attach Mode Refiner does not clean or verify any addresses it simply uses the Postcodes (already present in your database) to ‘attach’ or ‘append’ any of the data you select that is contained within Postcode Plus (such as Grid References). Clearly Attach Mode is only useful where addresses include accurate Postcodes. We recommend cleaning your database to obtain the best results and deliverable addresses. Attach Mode can also be used to process Postcode changes or to “back fill” address fields unique to a Postcode. e.g. a postcode which relates to a single address (like IM7 2DZ) will return a complete address; whereas a postcode which relates to a number of different addresses will return only the data common to all those addresses.

Please note that most advanced options such as de-duplication are not applicable when using Attach Mode. Field conditioning options (Settings tab) are limited to: specify the case (e.g. upper case or capitalised), maximum field length, and grid reference. These limited options are only available for the appropriate output fields.

Filling in the whole address from the property and postcode

You can also use Attach Mode to fill-in the rest of the address if you have data with only the postcode and property stored. It will match the property and return the whole address data. You should note that as the postcode is assumed to be correct the incorrect address may well be returned as postcodes can be mis-typed and the address has not been verified against any other elements. However in the case where you do not have any other data in your database but the property and postcode you may wish to use this to gain as many potentially correct addresses as you can.

To do this, select “Advanced Settings” from the “Settings” Tab and the “Assume original postcode is correct” option from the “Cleaning” tab.

Fields to use for Input and Output

From the Input tab map all fields which could contain the Postcode. Refiner will search all fields mapped for the postcode and look that up to provide the output data. Setting the name, organisation field etc. is therefore unnecessary for operation in this mode. If the Postcode is always in the same field only map that field.

From the Output tab map all the fields to be written back from a Postcode lookup. It may be helpful to map out the Result Code to identify which Postcodes could / not be matched. The Postcode itself can be mapped back out if you wish to write back any postcodes which have had typing errors corrected or which have changed following Royal Mail recoding.

Result Codes

The result code which is written to the Result Code field, if specified, in your database and displayed on screen on the Results tab. It is limited to the following values which differ slightly from their normal meaning:

Result codes 202-205 are only applicable if the ‘Assume Postcode Correct’ option is enabled. See the section ‘Filling in the whole address from the property and postcode’ above for more information.

  • 200 – Postcode was looked up and data attached.
  • 201 – Postcode was looked up and has changed. Data has been attached.
  • 202 Postcode was looked up and property data was also matched to it. Data has been attached.
  • 203 Postcode was looked up and has changed. Property data was also matched to it. Data has been attached.
  • 204 Postcode was looked up and non-matched property data has been added to the data attached.
  • 205 Postcode was looked up and has changed. Non-matched property data has been added to the data attached.
  • -1 – The specified Postcode was not found – it may have been removed from PAF or have been incorrectly entered originally.
  • -6 – No Postcode was supplied. Refiner examined all fields mapped in and did not find a Postcode in any of those fields.

Advanced Cleaning Settings

The Settings Tab (Shortcut Key [Alt]+4) includes an “Advanced Settings” button which allows you to configure some of the more advanced options related to Refiner cleaning.

In most cases the default settings are the best for cleaning data and will not need to be altered.

Note that you can reset Refiner cleaning settings to the defaults by selecting the “Reset To Default Settings” option on the File Menu.

Database Settings

These settings affect how data is written back to your database.

Retain any alias locality in the source address

Where an address is fully matched to PAF, Refiner will write back the full correct postal address to your database. Sometimes, your address data might contain a locality which is not postally required, such as “Wimbledon”, this will be removed should your address be matched as it does not form a necessary part of the address. Should you wish to retain such localities, as long as they are recorded by Royal Mail as a permisable locality alias, you can enable this option and Refiner will assume they are correct if given in the source data.

Never move source data to the Organisation field

Normally Refiner will put additional address data for street level matches in the property or building field unless they look like an Organisation or there is already a property. Specifying this option ensures Refiner never returns such data in the Organisation field – useful if you are not writing that field back to your databases data (the data will instead always be added into the property field).

Write back an empty string rather than null for blank fields

When using a Microsoft Access database or connecting to an ODBC source, it is assumed that where a field to be written out is blank a NULL should be assigned to that field as that is the normal convention with such database systems. However should you prefer, or your database system require, a blank string instead then selecting this option will force Refiner to output an empty string in such cases.

Write back a default DPS for street addresses

The DPS is a delivery point suffix, used by Royal Mail to uniquely identify an individual property on a postcode and included in the customer barcode which can be printed on an envelope. This is most useful to customers who are cleaning data for the purposes of using Royal Mails customer barcode mailsort products. Where an address can only be matched to street level, no DPS is avaliable and so none will be written out. By selecting this option a default DPS (9Z) will be written out for such addresses allowing them to be included in such sortations. However, please note that Royal Mail will be enforcing a minimum requirement that means that the vast majority of addresses in your data will need to have a full accurate DPS, not a default one, to be eligible.

Cleaning Settings

These settings affect the way in which Refiner matches your data.

Do not attempt to format fields for unmatched addresses**

Where Refiner cannot match an address it will attempt to logically format your source address into the output fields automatically. For example something ending Street will be placed in the Street field, something that looks like the postcode would go into the postcode field etc. If you are writing data out to the same number of fields as are present in the source address and the format of the original data is already good you may prefer to retain your existing address format in such cases. Selecting this option will cause Refiner to simply write out your source address data fields in order in your destination fields should it be unable to find a match (or leave it alone if you are writing back to the same fields).

Assume source data does not contain field separators

Refiner normally expects to find each address element in a separate field, for example the street in one field, the town in another, etc. Where this is not the case it would expect to find a separator, such as a comma, delimiting each address element in a field. Should these delimiters often be missing, e.g. you have the street and town all in one field with only a single space between each one and nothing to determine where one field starts and another one ends then using this option will allow Refiner to be able to match the address. Note that this option should only be used were your database requires it as the number of extra searches this requires means that speed of processing will be severely increased.

Match to PO Box only after trying a physical address match

Sometimes an address may contain elements of both a PO Box and a geographical address. In many cases only one of these may be correct and matchable, however sometimes they both could be and Refiner will attempt to match the PO Box address first in such cases. If you prefer physical addresses selecting this option will force Refiner to try and match a physical address first. Note that you will still get a PO Box match if no geographical address can be matched.

Assume original postcode is correct

Refiner uses advanced matching algorithms to find the correct match for your address while also ensuring that incorrect matches should not be given. In some cases your address data may have many fields missing, for example were only the property details and postcode have been captured, that means that very few addresses can be properly matched. In such cases if you are fairly certain of the accuracy of the postcodes held in your database you can select the option to direct Refiner to make an assumption that the original postcode is correct where a match cannot otherwise be made. This option is risky as you will obtain the wrong address should the postcode be incorrect so should only be used for non-sensitive uses of address data were it still has value even if some data is incorrectly matched. When used with attach mode you simply need to map in the property and postcode and the other fields will be attached based on those. When used with the default cleaning mode Refiner will still try to obtain a fully accurate match first, but if that cannot be made it will assume the postcode is given is correct and attempt a match on that basis.

Ambiguous Settings

When Refiner cannot make a unique match, an ambiguous one can sometimes be made instead, for user review. These settings allow you to switch off or adjust when such results are given.

Do not return ambiguous results (return no match instead)

Where Refiner cannot make an exact match, but can determine several possible correct matches, an ambiguous match will be given which allows the user to manually select the correct result if that can be determined through manual review of the record. In an automatic cleaning process the result code will indicate that the record is ambiguous but the original record will be retained, so this option makes little difference other than the exact result code returned. However if you do not wish to be advised of ambiguous matches you can enable this option.

Do not suggest un-verifiable matches (return no match instead)

In cases where in-sufficient address data has been supplied for Refiner to make an exact address, it is sometimes possible for Refiner to suggest a match due to their being a unique match to the data supplied. In such cases Refiner cannot confirm the match automatically as it may not necessarily be the correct one, but in the course of a manual process a user might be able too. Like ambiguous matches these will never be written back to the database in an automatic process, they have to be reviewed by the user. If you do not wish Suggested Matches to be returned you can select this option.

Ambiguous match is preferable to a street level only match

In some cases a match is ambiguous if matched to property level, but Refiner can return a street level match retaining the existing property information instead. This is normally preferred as with large databases the more that can be reliably cleaned automatically the more the savings in terms of user time. However, if you are manually checking records you might prefer to see the ambiguous matches in such cases to see if you can locate the correct result with human insight and so turning this option on will enable you to do so.

Grid References

This allows you to change the options for how Grid References and Latitude and Longitude values are returned.

Display Grid References

By default British grid references are used for British postcodes and Irish Grids for Irish postcodes. However in some cases you might wish to standardise on one or the other, for example when comparing distances between them. So, using Grid Reference Options you can select to always use either GB (British) or NI (Irish) grid references.

Where a grid reference is not available for a Postcode, there is an option to use an approximate grid reference for the locality or town. This is of course a lot less precise, but may be more useful than a blank grid reference for uses such as finding distances, locating an address on the Map etc.

Grid Reference Resolution

Unless you have purchased 1m grids, grid references are supplied to a 100m resolution. However you can choose if these are displayed as a 5 digit or 6 digit grid reference depending on the form that you need for the data. A 6 digit grid (usually 6 digits for both the northing and easting values) is capable of displaying 1m grids if required.

Latitude and Longitude Display

For latitude and longitude you can choose to use either the decimal value or a value made up of the degrees, minutes and seconds. Generally a decimal value is more useful for storing in your database to manipulate in other programs, but the degrees value is more easily readable when presented to an end user.

For more information about Grid References see the Grid References Appendix.

Programming With Refiner

The AFD Common API (Application Programming Interface)

AFD Products come with a fully functional easy to use Common API which enabled developers to integrate our products into most environments. The DLL’s required to access our data through the API ships as standard with all our products. A full SDK kit, which includes full documentation, example generation, and code to integrate into your own application is available for free download from our website.

Using our Common API you can develop for one of our address management products (e.g. Names & Numbers) and the same application will also function with our Postcode and Postcode Plus products.

For further information regarding our API, please visit www.afd.co.uk/support/

Appendices

Appendix A: Formatting a PAF Address for Printing

Overview

These notes are provided to assist developers using AFD’s Postcode Plus data via library calls, to re-create a correctly formatted postal address. Invariably, real-world computer databases seldom have sufficient fields available for the ‘official’ structure, so the Postcode Plus program makes sensible assumptions, whilst following these rules so far as possible. By accessing the data through the libraries, however, you have access to all the address elements in their raw state (except that the ‘Thoroughfare’ data already has its descriptors concatenated with the Thoroughfare name) so you can create your own address format as required.

A delivery point is composed of a combination of the following elements. Not all elements are present for each delivery point. Postcode and Post Town are the only two elements that are mandatory. County is no longer required as part of a correct postal address where the postcode is correctly given.

  
Organisation elementsOrganisation Name
Department Name
Premise ElementsSubbuilding Name
Building Name
Building Number
Thoroughfare elementsDependent Thoroughfare Name
Dependent Thoroughfare Descriptor
Thoroughfare Name
Thoroughfare Descriptor
Locality ElementsDouble Dependent Locality Name
Dependent Locality Name
Post Town
Postcode elementPostcode
PO Box elementPO Box Number
Organisation Name

If present, this should appear on the first address line.

Department Name

If present, this should appear on the second line.

PO Box Number

If present, this should appear on the first line if no Organisation/department details are present, else it should appear on the third line.

The following combinations of these elements may occur.

RuleSub-Building NameBuilding NameBuilding Number
1NNN
2NNY
3NYN
4NYY
5YNY
6YYN
7YYY

When a Building Name or a Subbuilding Name is present, then the format of the name can determine the position of the name in the address. (See Note).

Rule 1. Organisation Name

This condition may occur when an Organisation Name is used to uniquely identify a delivery point.

Rule 2. Building Number Only

The Building number should appear at the beginning of the first Thoroughfare line. If there is no Thoroughfare information, then the Building number should appear at the beginning of the first Locality line.

Rule 3. Building Name Only

Check format of Building Name (see Note).

If a Format 1 Building Name

the Building Name should appear at the beginning of the first Thoroughfare line, or the first Locality line if there is no Thoroughfare information.

Else

the Building Name should appear on a line preceding the Thoroughfare and Locality information.

When a building has a name and a number range, both must be held in the Building Name field because the Building Number field will only allow numeric characters to be held. Rule 3 will still apply to this.

Rule 4. Building Name and Building number

The Building Name should appear on the line preceding the Thoroughfare and/or Locality information.

The Building Number should appear at the beginning of the first Thoroughfare line. If there is no Thoroughfare information, then the Building Number should appear at the beginning of the first Locality line.

Rule 5 Sub-building Name and Building number

The Sub-building Name should appear on the line preceding the Thoroughfare and Locality information.

The Building number should appear at the beginning of the first Thoroughfare line. If there is no Thoroughfare information, then the Building number should appear at the beginning of the first Locality line.

Rule 6 Sub-building Name and Building name

Check format of Subbuilding Name (see Note).

If a Format 1 Subbuilding Name

the Subbuilding Name should appear on the same line as and before the Building Name.

Else

the Subbuilding Name should appear on a line preceding the Building Name, Thoroughfare and Locality information.

Check format of Building Name (see Note).

If a Format 1 Building Name

the Building Name should appear at the beginning of the first Thoroughfare line, or the first Locality line if there is no Thoroughfare information

Else

the Building Name should appear on a line preceding the Thoroughfare and Locality information.

End if

Rule 7 Sub-building Name, Building Name and Building Number

Check format of Subbuilding Name (see Note).

If a Format 1 Subbuilding Name

the Subbuilding Name should appear on the same line as and before the Building Name

Else

the Subbuilding Name and the Building Name should appear on separate lines

End if

The Building number should appear at the beginning of the first Thoroughfare line, or the first Locality line if there is no Thoroughfare information.

Note

Checking the format of Subbuilding Name or Building Name

Certain of these names can appear on the same line as other address elements. These are called Format 1 names in the address formatting rules. They are identified by:

i. First and Last characters of the Name are numeric

ii. First and penultimate characters are numeric, last character is alphabetic

This covers names like:

2A

123-126

81 & 85

Appendix B: Command Line Options

The following entries can be added as part of the ‘Target’ or ‘Command Line’ when launching Refiner, allowing you to specify your preferences for a particular use of the program and automate the cleaning process:

  
/Minimise=YesRefiner will load minimised when this is specified
/Session=default.rfrSpecifies the Session File to be loaded if ‘default.rfr’ in the Refiner folder. You can specify the full path to the session file if desired. By default Refiner opens the last session used.
/Auto=1Providing the database opens correctly an automatic process runs on the database as soon as Refiner loads. Please note this requires the session to have been correctly set-up with the processing options you wish to use first.
/Auto=2As above but Refiner exits following the auto process run
/Splash=NoSuppresses the display of the Splash Screen when loading
/Tab=6Sets Tab 6 (the Results Tab) to be displayed after loading. By default Refiner opens on the last tab used.

These commands can be executed from the Start Menu, Run by typing something similar to the text below into the the Run box. . .

c:\pcplus\refiner.exe  /Splash=No /Session=Default.rfr /Auto=1 /Minimise=Yes /Tab=6

Appendix C: Keyboard Shortcuts

FunctionShortcut Key
Automatic Process[Ctrl] + A
Begin Processing Records[Alt] + P
Borrow current record from Postcode Plus[Alt] + B
Cancel Processing[Alt] + C
Censation Tab[Alt] + 7
Copy current field to Clipboard[Ctrl] + [Insert]
Copy full record to clipboard[Ctrl] + F
Close Refiner[Alt] + [F4]
File Menu[Altl] + F
Help[F1]
Help menu[Alt] + H
Input Tab[Alt] + 2
Open Database[Ctrl] + D
Open Session[Ctrl] + O
Open Tab[Alt] + 1
Options Menu[Alt] + O
Output Tab[Alt] + 3
Paste current Clipboard contents to field[Shift] + [Insert]
Paste full record from clipboard[Ctrl] + H
Process Tab[Alt] + 5
Results Tab[Alt] + 6
Save current settings[Ctrl] + G
Settings Tab[Alt] + 4
View Menu[Alt] + V

Appendix D: Grid References in AFD Products

Grid References

Grid Reference data is primarily sourced from Ordnance Survey for distance calculations, nearest calculations and data / location analysis. It is made up of a six digit Easting and a six digit Northing. This reference relates the location of the Postcode to the National Grid (or Irish Grid for Northern Ireland Postcodes (start with BT)).

Background to Data

England, Wales and Scotland

The largest units of the National Grid are 500km squares, each designated by a prefix letter – the first letter to be quoted in the National Grid reference.

The 500km squares are then broken down into twenty five – 100km squares; these are also designated by a prefix letter – the second letter of the grid reference.

Within the 100km square, each smaller grid square (at 10km, 1 km, or 100m side) is designated by the distance of its South-West corner from the West (Eastings) and South (Northings) margins, respectively, of the 100km square in which it lies.

To enable these distances, or co-ordinates, to be easily found, their values are printed in the map margins against the grid lines concerned. These provide the numbers – two, four or six digits – which identify each particular grid square.

EXAMPLE
e = easting     n = northing
S   K   1   2
e   n

shows a point 10,000m East and 20,000m North of the South West corner of the SK square.

S   K   1   3   5   2   4   6
e   e   e   n   n   n

Shows a point 13,500m East and 24,600m North of the South West corner of the SK square.

A point or feature, within one of the smallest grid squares, can be indicated still more closely by estimating the tenths of the appropriate grid square.

When giving a National Grid reference, for any point, the distance Eastwards (Eastings) is always given before the distance Northwards (Northings).

Northern Ireland

Northern Ireland is covered by the Irish Grid System, which contains a single alpha character to represent each 100km square. There are 5 such 100km squares covering Northern Ireland using the letters: C D G H J.

This, however, makes the distance difficult, so AFD products automatically convert the Irish grid system to British system to allow distances / nearest to be correctly calculated. You can opt to convert all grid references to use the British or Irish Grid systems to provide consistency across the whole UK and these options are available in the front-end and developers can gain access to these conversions through the Common API.

Latitude & Longitude

To enable these Grid References to be used with GPS systems and other tools requiring latitude and longitude values AFD Products provide a conversion to latitude and longitude values for a grid reference which is displayed in the front end and can be inserted into your application. Developer’s can gain access to these conversions through the Common API. The latitude and longitude values given are based on the WGS84 standard – the one in most common usage with GPS systems. The conversions used are accurate to within 5 metres (the worst case scenario at locations with most distortion). But remember that a grid references at postcode level may cover a wide area especially in rural areas.

Appendix E: End User Licence Agreement

See https://www.afd.co.uk/eula

Appendix F: Re-registration Form

See re-registration form

Appendix G: Royal Mail Terms

See www.afd.co.uk/rmterms

mailLink mailLink

We are here to help

We serve thousands of organisations and a network of hundreds of partners across multiple industry sectors, enabling them to have full confidence in their contact data.