Handwriting recognition - has anyone found a robust way to recognise and convert to text?

[KarlMelrose]

Dear Community,

Has anyone found a robust solution for recognising and converting handwriting in scanned documents to text - or at least making it searchable?

I’m looking to solve a problem with septic tank inspection forms in NSW local government. They are tremendously labour intensive to process. The inspections are carried out by licensed independent contractors who all use their own forms. There is a certain amount of required information, but details like address etc. are handwritten, and the original is provided to the property owner and a secondary carbon copy provided to the council. Each form then needs to be checked for compliance, sent for enforcement action if risk factors are identified, and tagged against a property file, or as a worst case, at least have an OCRd rendition saved so that it’s searchable if there is a need to find it later. It’s massively labour intensive at the moment for no improvement in service level delivered, and it is very hard to identify non-inspectors.

Robust handwriting recognition could provide a good bottom up solution that would solve the handling problem.

I’m also talking to the legislative authority to see if there’s a top down way to fix it, but these are rural areas so web forms and those sorts of technologies aren’t really usable.

I’ve spoken to vendors who might traditionally provide this technology - abbyy, ezescan, docscorp - however most approaches require each letter to be contained in a box individually. These forms are lines on paper (I’ve attached particularly horrendous sample). We are also about to start testing the Google Vision API and have seen some preliminary announcements of Microsoft Cortex that look promising - but unavailable for a significant period of time.

Has anyone round a reliable and robust solution to handwriting recognition?!

IMG-0343%20(1)|375x500

Thanks!
Karl.

[Sanderson]

Hi Karl
Can’t help you with handwriting recognition problem. However, the annual processing of many thousands of on-site sewerage system maintenance certificates are a problem for many! And it may be worth looking further at what others are doing.

I know you mention that these are rural areas, and I know that it’s difficult to use some apps when there is no internet connection. However, other councils also have to deal with the problem of remote areas with no internet connection for these types of maintenance certificates. While the inspections themselves might be in remote areas, most businesses maintain some kind of internet connection at the office, so using some sort of app or online lodgement is not impossible when it comes to passing that information on to government.

Gympie and Noosa trialed an app called OMA, which was successful.

Sunshine Coast Council liked the concept of OMA but also wanted to allow businesses to use their own software and went on to set up a system with their eServices platform for electronic lodging. It looks like owners nominate a particular service agent who is then connected to all the details for that property and then report against that property each time they do an inspection. Something like this might work in remote areas (if the app has the property details, and then the details are just uploaded once they are connected to the internet).

Here at Townsville City Council we are also looking to develop online lodgements. We also have the problem that some areas don’t have internet access, so our process has to take that into account.

Hope this helps point you towards something you might investigate further, although I know it’s not the question you were asking.
Sheree Anderson

[KarlMelrose]

Thanks Sheree - that’s amazingly helpful. I think it’s a good case of “ask for the outcome, not the method”. Ultimately, I’m looking for a way to solve the problem of septic tank inspection forms not being kept/available as records and then failure of all of the compliance processes that goes with that - so this is perfect. I’ve spoken to half a dozen councils in NSW but none of them were aware of other ways to solve it - they just have the same stacks of forms appearing on a regular basis (or I’m talking to the wrong people).

It sounds like OMA doesn’t have an answer for offline? That’s a problem or most of the councils I’m talking to as they’re all rural. I’ve found OMA and I’ll have a chat to them about it - thanks for the pointers.