Mapping Ideas with Open Source Handwriting-to-Text Software

hey there world

Ideas are curious things. You’re going along and suddenly… Boom! An idea. But it wasn’t really out of the blue, was it? Sure, something immediate may have sparked it. The spark came from somewhere. Where is that? How.. What is an idea? Configurations of neurons grow and represent realization..such vague explanation is saying Earth is a space rock. It is this. It’s immensely more.

Through language and emotion we navigate the world. We choose how to spend time, with whom to spend it. We choose which ideas are most interesting and seek out more. We read, browse, share. Videos, politics, religion, business, creativity.. our modern world is full of inspiration for those who explore. The idea I’ve been most interested in exploring is categorical: pondering how it is that ideas come about in the first place. Along this intellectual journey, I encounter a great breath of people, perspectives, and experiences. Playfully I explore the world of minds in order to create a perspective that is uniquely this self, this Amy.

In an effort to explore a bit of how I became myself, I’ve developed a side project over the past few years: mapping ideas. This post and accompanying a lecture at Stanford summarize the why.

Let this focus on the next step: sharing.

I write frequently. Between 2008 and June, 2016, I have filled up over 40 Moleskine Notebooks. Words, musings, aphorisms, speech outlines, sketches.. these notebooks are not mere writing, if there is such a thing. They are the evolution of the ideas I hold dear. The ideas I think have played the lead roles in my becoming who I am today.

Analyzing language for things like sentiment and keywords has for many years been possible thanks to platforms like AlchemyAPI from IBM. There is still a missing puzzle piece: a tool that automatically transcribes images of notebook pages into text files. Several tools exist but none so far have worked for me. I realize that Evernote makes writing searchable. It does not allow you to export text files.

Anyone out there interested in computer vision tools that turn writing to .txt? I would love to team up with you. Not just to make the next step in mapping ideas but to create an open source tool that anyone who writes can use to transform handwriting into a digital database.

I’m @amyleesterling on GitHub and Twitter. I’m still of course ever-searching for deeper understanding of who I am and perpetually welcome conversations with those of you on similar journeys. Ping me if this is up your curiosity alley. These are my personal notes and currently I’m brave enough to share them with people who ask but not publicly on the web just yet 🙂

A few sample images of notes below illustrate content diversity and wackiness.

Also published on Medium.

Also explore..


  1. Thanks, you literally just sidetracked me for half a day, in a good way. Your article inspired me to flip through my own Moleskins, and other various forms of notebooks, that I’ve filled throughout my years. A collection of 40+ is certainly an impressive amount of writing, note-taking, artistry, etc in an 8 year span. I recall a couple of years ago exploring various methods of scanning/photographing to pdf/ocr/transcription, but I did not find any silver bullets to do the job. With crowdsourcing, ie mechanical turks, being so accessible and affordable, it’s probably the most accurate and in the end pretty affordable. One challenging aspect was capturing all the doodles, technical drawings, side notes, that couldn’t be captured without curated description, so again the turks came to mind. Sounded great and a fun project, but I didn’t have the time to pursue development of this pet-project at the time.

    So fast forward to today, after reading your article, which reignited my interest, I went on another scavenger hunt and found a few new and interesting leads and tools I hadn’t come across before.

    Seems a company called Captricity has accomplished much of what I was envisioning with the crowdsourced recognition in combination with OCR. Some really clever stuff and applied it to some important disciplines (healthcare, insurance, govt, etc). Other tools I found support the creation of a custom developed solution using manual annotation tools, thinking that if I’d ever wanted to set-up my own software to support a Turk farm. Not sure which path is the least resistance. I’d start with the Captricity, maybe they have an affordable solution for notebook enthusiasts, and if not, they should. 🙂

    Here are a few interesting links I found during my quick research binge (in no particular order): (this is pretty much the answer to your notebook challenge) (crowdsourced searchable art annotation) (crowdsourced madical imaging annotiation) (connect it to a searchable cloud db) (connect it to a searchable cloud db) (real people)

    Until I have more time to explore this interesting effort, I am pressing pause to get back to my regularly scheduled stuff.

Leave a Reply

Your email address will not be published. Required fields are marked *