Deep learning

Intro

I don’t know where this post will lead to (started the journey 01/21/19). The commitment is to get a Neuronal network add-in for Excel… limit time is end of 2019 (nothing bad would happen if goals could be achieved earlier). Absolutely, Excel is not the right tool to carry serious ANN -in terms of perfomance-, but, considering that all the computations can be shown in real time, and that you can mount a RAD model of every architecture, it should be considered among the better options to learn the subject from the basics. And the two final reasons where I finally arrive at: because I can do it, and because is so ubiquous that it does not limit to matematicians and IT people to play this things. If you want to make big stuff, you should consider other platforms and even other hardware… but that’s not my target and this is not your site if you have arrived here looking for other than Excel. Going to the business, from all the architectures that can be performed under the label of ANN, I’ll try to first pursue the ConvNet as they expose most of the algebra needed to perform the whole thing, so I hope it will be easier for me to adapt to other type of architectures. The final goal is to get a tool that can be good for any ANN model (considering the size limitations). This post should end fused with my previous post on Neuronal Network on Excel (task pending…).

Milestones

Expecifically I’m looking after the following goals:

• NN activation functions (those exposed on the wikipedia article). Achieved 01/22/19
• NN matrix algebra (multiplication, add/substraction, element-wise operations… with matrices/vectors). Half achieved, refactoring the MatLab translator module. Others should came from the JS translator.
• NN neuron manipulation (add/delete/copy/move).
• NN layer manipulation (add/delete/copy/move).
• NN specific functions. Filter, Pool, stride, convolute, backpropagate,…
• BMP/JPG import/export… someway achieved refactoring the ASCII Excel module.

As soon as each goal be achieved, then it will be converted to milestone. There’s one specific problem if this is to be done under Excel Environment… limits. For example, the >2K3 column limit of 16384 can hardly allow to show/represent more than a set of (72×72, RGB) bits in vectorized form in one worksheet. I still don’t have managed a way to overcome this limit, as I consider necessary to show the data on the cells, but surely will come to one (split neurons on different worksheets, aggregate inputs in 2D shape,…).

Other’s implementations that I can base the thing on…

Not an exhaustive list, and maybe not any usefull, but here is a recopilation of other software I’ve seen to perform NN tasks:

• SNNS (Stuttgart Neural Network Simulator)
• IMPLEMENTATION OF AN MS EXCEL TOOL FOR BACKPROPAGATION NEURAL NETWORK ALGORITHM IN ENVIRONMENTAL ENGINEERING EDUCATION (Selami DEMİR & others)
• XLStat
• deepExcel (even it’s satiric)

Bibliography

• Andrew Ng specialization course -videos- Deeplearning.ai, on Coursera platform. Very instructive, tips are explained in a very clear form, so anyone, even with not a high level of maths, can follow. You need to know and understand algebra. Only one handicap, IMO, is that it’s focused on TensorFlow (Google) platform.
• Jeremy Howard’s courses on fast.ai platform, online courses -also on Youtube– (and here a little trick if you’re to follow this course). To use on PyTorch, that seems to have its momentum by the end of 2018 and as a true OpenSource alternative can stay for long. Differences between them spotted here.
• Geoffrey Hinton Machine Learning course (Collin McDonell’s youtube list). You will need a stronger math basement to follow this lessons.
• Make Your Own Neural Network in Python book, by Tariq Rashid
• Ian Goodfellow/Yoshua Bengio/Aaron Courville MIT book. A lot of algebra there (maybe a hard path to go if you’re to learn about NN).
• Christopher Olah’s blog.
• Andrej Karpathy’s blog
• Andrew Trask’s blog
• Emill Wallner’s blog and all the other stuff on FloydHub
• https://pyrenn.readthedocs.io/en/latest/create.html
• https://medium.com/@geek_kid
• …
• Youtube. You can drown before getting bored or quit out, so here is my ANN channel list on the topic to have all the videos I liked in the same place.

Other info

• Short explanation on ANN and  introduction to CNN. From same author, his repository on GitHub.
• Another explanation, from the basics.
• Activation functions:
  • https://medium.com/@himanshuxd/activation-functions-sigmoid-relu-leaky-relu-and-softmax-basics-for-neural-networks-and-deep-8d9c70eed91e
  • https://medium.com/the-theory-of-everything/understanding-activation-functions-in-neural-networks-9491262884e0
• Convolutional Networks (ConvNN or CNN)
  • Begginers class on CNN
• Capsule Networks (CapNN)
  • https://github.com/sekwiatkowski/awesome-capsule-networks
  • An interesting question on Quora: https://www.quora.com/Will-capsule-networks-replace-neural-networks
  • https://hackernoon.com/what-is-a-capsnet-or-capsule-network-2bfbe48769cc
  • https://www.oreilly.com/ideas/introducing-capsule-networks
  • Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton. Dynamic routing between capsules (PDF). November 2017.
  • Geoffrey Hinton, Sara Sabour, Nicholas Frosst. Matrix capsules with EM routing (PDF). April 2018.
  • Geoffrey E. Hinton, Alex Krizhevsky, Sida D. Wang. Transforming auto-encoders (PDF). June 2011.
  • Aurélien Géron. Introducing capsule networks. O’Reilly Media, Ideas, February 2018.
  • Max Pechyonkin. Understanding Hinton’s capsule networks series. AI³, November 2017.
  • Nick Bourdakos. Understanding capsule networks — AI’s alluring new architecture. FreeCodeCamp, February 2018.
  • Thibault Neveu. Understand and apply CapsNet on traffic sign classification. Becoming Human, November 2017.
  • https://nips2018vigil.github.io/static/papers/accepted/7.pdf
• GANs:
  • http://guimperarnau.com/blog/2017/03/Fantastic-GANs-and-where-to-find-them
  • http://guimperarnau.com/blog/2017/11/Fantastic-GANs-and-where-to-find-them-II
• HTM
  • HTM School

Moreover, some other projects on GitHub:

• Jean-Carlos Paredes, super simple sample of how to implement a decoder for MNIST dataset in Excel (just for the number “7”).
• Mike Pallister’s
• https://github.com/vrishank97/Introduction-to-Machine-Learning
• Excel NN

On Medium there are a lot of interesting articles as well (I must confess that I personally get surprised of how people put a lot of creativity to achieve a goal that stands above most “expert” no-sayers):

• David Smith, really nice implementations in Excel, part1, part2
• João Telo
• Emanuele Bonura, look at https://medium.com/@bonura.emanuele.sv/a-neural-network-in-11-lines-of-vba-4367a7219441, that has follow from this.

No Python performance on Excel?. Don’t worry, use Python in Excel and get the best of both worlds. Even on Google Sheets…