Replace algorithms with NN
Replace algorithms with NN
This idea of replacing software components with neural networks comes
to us, or atleast me quite a few times. I remember now the very first
time I thought about this. At that time, I did not know anything about
neural networks other than that they are a part of Aritifcial
Intelligece. I and my friend Suriya were experimenting with lots of
stuff that we can get our hands on. Arduino, OpenCV, Matlab, Linux,
device drivers, etc. Yes it was five years ago when we came up with the
project
"Adhi"(means beggining). It was about intergraing AI, NN specifically
into linux kernel to make it kernel smarter.
There is another one,where I imagined a symbolic AI and nerual
network will compete to become better than each other. This came to me,
when thinking of how to adjust what a network learns. Symbolic AI are
easier to tweak.
Like this one, most of the ideas presented below are old and may be
useless, and I admit, these are all just vague phrases and I never made
an attempt to implement them. But I hope this time I will get my hands
dirty and this would at least serve as a reference.
Why am I writing about this now?
One. I have been planning to write a software to help with
tamil-linguitics. There is a lot to discuss in it, But I will limit
myself to one particular area - WordNet. Two. few months ago I wrote
about how computers can be more useful that being just a data storing
searching devices. In that post I tried to explain how we are
underutilising out computers. I always try to organize the directories
I have on my computers to aid the way I work. And there are some more
idea, and a combination of those things led me to write this.
In this post, lets discuss a specific algorithm - indexing. Why? No
reason. It is way more easier to explain, I guess.
What is an index?
Let's say you own stationery store, which sells, papers, notebooks
and pens. They way you place those items your shop is essential for
business. Pens are almost always placed at the front. Notebooks are
placed in wooden or iron racks and are organized interms of their
attributes - ruled/unruled, long/short, number of pages,
hard-bound/paperback and etc., and other items which are not frequently
sold will be kept somewhere in the back. Basically, we keep the
frequently sold items closer and the others farther. We can think of
this, as an indexing problem can't we?. The next time we buy notebooks
to sell, we place them accordingly.
Indexing and related data-structures:
Binary-tree can be thought of as decision tree, but it is not as
sophisticated a decision tree.We, humans out of our lessons came up with
B-tree. TODO
We can think of programming in general as way to give structure to
information/data.That is what we do right? We take in bunch of data and do
something with it. Before doing with it we need a idea of what it is.
A remotely relevant example: Games
Let's take games. Games are one of the crucial softwares. They are
complex and compuationally intensive and some time they look incredibly
real. The more real they look, the more computations it has to perform.
The real world has structure. A computer sits on a table. The table mostly
stays inside the building. Our hands holds items. Bikes and cards rides on
the roads. There are numeraous relationships between objects in the real
world. And there is always more in the real world that what meets the eye.
But we cannot afford to make the computer build an entire world inside.
The real world is too rich in complexity to be fit in 32Gb RAM, and the
physicsal laws are too intense to calculated with a quad core processor.
So what do we do? We cull off the things that don't meet the eyes and
display only what is necessary on the screen. How do we do it? Similar to
real world we simulate a small scale world inside the computer with help
of 'relationships between items'. Scene graph it is called. It is the data
structure that contains the objects and their relationships, including the
player from whose POV we see the world inside the game. So that the
computer can evaluate what object falls under our view, and only act upon
them, i.e apply physics over the objects in the view so that we feel
immersed in the game. To make it more closer to real world. The people who
create games optimize the game and its engines to run as fast as it can in
as many hardware platform as possible.
Different kinds of games build different kind of worlds. GTA Vicecity
was one of the first open world games I played.But mostgames are of closed
world nature. i.e you cannot move around freely.There are only a set of
paths you can go and only a set of things you can do. This in a way
reduces the complexity in the computation.
Have you ever wondered why some games are fast even at high graphical
quality settings and some games suck even atlow quality settings. That
depends upon whatoptimization havebeen done and whohasdone it.Naturally a
game programmer with good experience can optimize their games better than
one with lesser experience.
A extrememly remotely relevant example: Software
The sotware we use, Microsoft word or Libre office Writer, are
general purpose software. Almost every one will have some use for it.
There is a set of usual things we do with a computer. Write docs, paint
pictures, watch videos and such. But there are also specialised
softwares designed for specific purpose. For example banks use custom
designed software. The functionality of software is tailored its
industry. You need measurements to tailor stuff.
A relevant example: Out stationery store
The way you store items in you shop will differ from other shop, even
if it was a stationery store right next to yours or right opposite. Why
it dependsupon so many factors. The kind of items you sell, the number
of items in each kind, the amount of space you have etc. i,e how you
store, depends heavily on what you have. Index is a function of data.
Let's think about, database coders specifically people who code the
indexers. They cannot make any predictions about what kind data would be
stored in the databases other than few bare essentials like integers and
string. They cannot know what kind of data that I am going to store. It is
this disconnect which penalises performance.
Coming back to WordNet
Wordnet is like a thesaurus but richer in information. Building a
wordnet is no simple task. It is not exaggerating, when I say it might
take one's lifetime to build one. How can we make use of computer in
this endevour? This is a crude example and by no means intended to be
comprehesive implementaion note.
Let's say you have software, you lookup a word and it shows some
information regarding that word. Its meanings, etymology, different of
forms of use in sentences, etc. What if it can suggest a list of words
and you can edit the relationship between them, i.e if the suggested
list of words are closer to this word in some sense? Wouldn't it be
awesome?
But how can we pick a list of words from million words closer to the
one that we are reading? Well what if we can store the words in database
based on their similarity and relationships with other words? i.e store
the words closer in meaning, closer in memory.You mayask how can we know
what words are similar and what are not?
That is where machine learning comes in. There are machine learning
methods, word embedding for example can be used to come with a
similarity metric. And manual contributions via this linguistic software
inturn can be used to make it more accurate. Remember the meaning of the
words are always changing, and there is not absolute meaning and
relationship between a word and the meaning it represents. So this
process will always be incremental and iterative.
The point is, neural networks and other machine learning methods can
be use to determine where and how to store words.Unlike a general
purpose database, the neural network will learn what is in the data so
that it can store the data more efficiently and effectively.
I imagine a bunch of AI agents talking to each other can do a better
job at this than a handful of coders.
Comments