The human brain does two functions very well – the first is that it can recognise images very well. The second is that it can detect patterns based on time and make predictions based on those patterns.
The last few months I played around with the first function- — around building image recognition neural networks (CNN)– the ones that help you determine whether a picture is of a dog or human (source code) or another example is detecting faces in a picture (article) or recognising handwritten numbers (article).
For example, we can write a CNN that determines that the above picture is of a horse.
But, if you ask a CNN to tell us if this horse is running or not — it would have a hard time. If you ask a human — the answer is dependent on a number of things and most on time. If you saw this picture as a series of pictures where the horse was loitering around, you could make the determination that it is likely that the horse is standing around. In that case, we used human memory to solve the problem of determining what the horse is doing.
Recently, I started to dig into the second function of the human brain and see how you can mimic the human brain. To answer a question — how do you introduce an element of time in AI.
The canonical example is that I would like to predict the prices of a stock. Another example is Google Assistant, it needs to understand the context to service a request.
Recurrent Neural Networks (RNNs) and Long Short Term Memory (LSTMs) help you solve problems that have dependencies on time. I have blogged (here and here) about LSTMs previously so lets take RNNs in this blog.
The following picture is called an unfolded model of a RNN.
Let’s break this picture down.
Each circle is a feedforward NN (ffnn) which means it is taking input (i), does some calculation and puts out a value (o). The weights calculated by the ffnn are Wi and Wo on the input and output respectively. NN’s typically take anywhere between 1 to thousands of inputs, thus, i can be in thousands. NNs typically output 1..n outputs as well, thus, o can have that range as well.
So far so good.
Let’s now bring in the notion of time.
RNN’s keep an internal state around (s) and Ws are the weights that are generated for that state. Think of s as the memory component of the RNN.
The way you bring in the memory element along is that you feed the memory from time t as the input to time t+1 and so on. This, is how RNNs are different than standard feed forward networks. The input increases from i to i+s.
s_t = some_function (Wi*i + s_t-1*Ws)
The picture is that of a simple NN repeating itself over time starting with t upto t+2 but really going on to infinity.
The next level of complexity is to stack an RNN on top of another and create a 2 layer RNN. Two layers is the beginning because you can stack an arbitrary number of them.
This lattice of RNNs then provide the flexibility to process temporal dependent data and make complex predictions.
About 3 months back, when I heard that Prem uncle passed away in his sleep, I broke down and cried. I cried like I haven’t in a long time. His passing away has left a huge hole that can never be filled.
I was 18 when I had my last substantial interaction with him. I must have met him about 3-4 times since I left Mumbai in 1992 and 26 years later, he (and his wife) still remains one of my favourite human beings. For that matter, his parents were fantastic human beings too.
I got thinking in the last month or so – In a fast moving world, filled with short interactions — why do I miss him? Why was he special? What are the lessons that I can bring in my life and hopefully in yours too from him? What made him an extra-ordinary human being?
He wasn’t my teacher, relative but a neighbour who became much more.
Prem uncle and Sangeeta aunty, lived in our apartment complex were in their 30s when we met them. Presumably, I saw him at his peak — financial and health wise. He had a fantastic business going, the confidence that comes along with having made it and made it very fast. As a 16 year old, going down his career path was something that I seriously considered.
Then, it came crashing down — he had to go through two heart valve replacement surgeries, one anticipated (and I think triggered by chain smoking) and another one because the first valve was defective. Followed by a paralytic attack. Between these three events, he suffered a betrayal by his business partner who pounced on the opportunity to take over the business and drove him out. All of this in a span of 2 years :-(. Having recently suffered a betrayal myself, I can truly empathise the pain and the hurt that he had to go through.
The financial success then wasn’t what made him extra-ordinary. It isn’t his material success in the world that mattered to me. The people who hung around him who craved the reflection of his material success evaporated very quickly.
Here are the lessons that I learnt from him:
As I got thinking, I realised, the reason I loved him, or my sister loved and now his son and nephews and nieces loved him was because of unconditional love and acceptance that he radiated towards us . I truly have never seen any other human as generous as he(and aunty) was in sharing his love. And when he accepted you in his life — he truly well accepted you for all the good and the bad.
When he interacted with you, you were truly the centre of his attention. There were no half measures in terms of looking at others. This was important for a 15 year old kid — a human/an elder paying complete attention to me and acknowledging my viewpoints.
Bringing joy and fun in every interaction with friends and family. Be it the nightly carrom sessions or the hours of Nintendo games or forcing me to play Antakshari (an Indian singing game). The focus was truly to enjoy life and bring joy to everyone involved. To this date, when I miss him, I reach out to my Nintendo and play it. Consequently, I never play antakshari because it reminds me too much of him and wonderful time gone by that can never be recaptured.
Playfulness in relationships. I loved how he brought play in relationships. Minor irreverence perhaps — not the right word perhaps gentle ribbing are the right words. A minor playful drama with his wife, father and mother pretending to hide his smoking habit while it was out there in the open. Jumping in the car to go get paan in the middle of the night or for that matter getting me to sit on the hood of the Jeep and drive in the beach. It was all great fun.
That’s really it! Living a well lived life doesn’t need trappings of success, huge amounts of money or too many accomplishments.
Looking back at his life and seeing how well I embody those lessons. I find that I fall woefully short. Life today has become too serious, attention fragmented by devices perhaps a small bit of playfulness is what I embody from his life. His passing away has been a wake up call for me to refactor my life to the qualities that I admired in him and admire in others.
It is worth repeating that a life well lived doesn’t need trappings of success, huge amounts of money, too many accomplishments, status but what it needs is unconditional love, joy, fun, playfulness.
Prem uncle – you will be forever missed but you will continue in our hearts. Thank you for all the fun times and the love and congratulations on a fantastic life and life well lived!
In the last blog, I gave an overview of LSTMs (long short term memory) in AI that mimics human memory. I will use this blog to go 2 layers below to draw the building blocks of this technology.
As a reminder, at a 50k level, the building block looks like the image on the left. There is long and short term memory on the left –> some input comes in –> new long and short term memory is output on the right. Plus, an output that determines what the input is.
Let’s open the box called the LSTM NN (neural network). This block is composed of four blocks or gates:
- The Forget Gate
- The Learn Gate
- The Remember Gate
- The Use Gate
The intuitive understanding of the gates is as follows: When some new input comes in, the system determines what from the long term memory should be forgotten to make space for the new stuff coming in; this is done by the forget gate. Then, the learn gate is used to determine what should be learnt and dropped from the short term memory.
The processed output from these gates is fed to the remember gate which then updates the long term memory; in other words a new long term memory is formed based on the updated short term and the long term memory. Finally, the use gate kicks in and produces a new short term memory and an output.
Going a level deeper:
The learn gate is broken in two phases: Combine –> Ignore.
- Combine: In the combine step, the system takes in the short term memory and the input and combines them together. In the example, the output will be Squirrels, Trees and Dog/Wolf (we don’t know yet — see previous blog for context)
- Ignore: In the second phase, information that isn’t pertinent will be dropped. In the example, the information about trees is dropped because the show was about wild animals.
The forget gate decides what to keep from the long term memory. In the example, the show is about wild animals but there was input about wild flora, so the forget gate decides it is going to drop information about the flora.
This gate is very simple. It adds output from the Learn gate and Forget gate to form the new long term memory. In the example, the output will be squirrel, dog/wolf and elephants.
This gate combines the input from the learn gate
Math behind the various gates for mathematically inclined
Combine phase (output = Nt)
Take the STM from time t-1, take the current event Et and pass them through a tanh function.
Nt = tanh (STM_t-1, Et)
Mathematically Nt = tanh (Wn [STM_t-1, Et] + bn) where Wn and bn are weight and bias vectors.
Then, the output from the combine phase is multiplied by another vectory called i_t that is the ignore factor from the Ignore phase.
Ignore phase (output = i_t)
We create a new neural network that takes the input and STM and apply the sigmoid function on them.
i_t = sigmoid ( Wi [STM_t-1, Et] + bi)
Thus, the output from the Learn gate is:
tanh (Wn [STM_t-1, Et] + bn) * sigmoid ( Wi [STM_t-1, Et] + bi)
The forget output is calculated by multiplying the long term memory with a forget factor (ft).
The forget factor is calculated using the short term memory and the input.
ft = sigmoid( Wf [STM_t-1, Et] + bf]
Forget output = LTM_t-1 * ft
The remember gate takes output from the Forget and Learn gates and adds them together.
LTMt = LTM_t-1 * ft + Nt * i_t
The use gate applies a tanh function on the output of the forget gate and multiplies it to a sigmoid of the input and the event.
STMt = tanh (Wu [STM_t-1, Et] + bu) * sigmoid ( Wv [STM_t-1, Et] + bv)
The four gates mimic a point in time memory system. To truly envision this, think of a lattice of connected cells separated in time. Thus, the memory system will continually evolve and learn over a period of time.
(credit: the math and the example are coming in from the Udacity deep learning coursework)
Memory is a fascinating function of the human brain. Specifically, the interplay of the short term memory and long term memory where both work in conjunction to help humans decide how to respond to the current stimuli is what makes us function in the real world..
Let’s take an example —
I am watching a program on TV and suddenly a picture of a dog/wolf comes up. From the looks of it, I cannot distinguish between the two. What is it – a dog or a wolf?
If the previous image was a squirrel and since squirrels are likely to be in a domestic setting, I could make an assessment that the current image is a dog.
This would be reasonably true, if all I had was my short term memory.
At this point, my long term memory kicks in at this point and tells me that I am watching a show about wild animals. Voila, the obvious answer at that point is that the current image is of a wolf.
LSTMs mimic human memory
A specific branch under Deep Learning in AI called LSTMs (long short term memory) are used to solve problems that have temporal (or time based) dependencies. In other words, LSTMs are used to mimic the human memory to predict outcomes. Unlike another branch in Deep Learning called RNNs (recurrent NNs) which only keep the short term memory around, LSTMs bring in long term memory to bring in high fidelity to their predictions.
The Working of LSTMs
- What should it forget? Trees in this case because the show is about wild animals and not trees.
- What should it learn? There is a Dog/Wolf in addition to the squirrels and trees.
- What should it predict? The Wolf
- What should it remember long term? Elephant, Squirrel, Wolf
- What should it remember short term? Squirrel, Wolf
All the above is done for the current time and the new long/short term memory are fed into the next input that comes at time t+1.
Thus, you can think of the above picture as recurring for every time epoch t.
In the next blog, we will deep dive into the LSTM NN and see how each of the bulleted questions are answered.
Pretty interesting, isn’t it?
(disclaimer: the example used is from the Deep Learning course work on Udacity)
(If you rather see the pictures, here they are)
Days 3, 4 and 5 in Santorini – Oia, Imerovigli and rest of Santorini
Describing Oia in a blog post is very easy – we went to Oia and stayed there for 2 days :-). Because there isn’t anything else you do in Oia — you get there and enjoy it.
On the other hand, Oia is a place where tourists line up 3 hours before a sunset to see it. Often, tourists duke it out to get the best picture of the sunset. Chinese brides (more in another blog) walk around in their wedding dresses to take their dream wedding picture.
Beauty against the contrast of heavy cardio workout in a hot, humid environment is how I remember Oia.
Oia is not for weak knees, you are either climbing up or you are going down and that in the humidity of the island is a killer. What is amazing is that each hotel has a porter that makes this trip multiple times a day with bags on their shoulders or on a dolly.
Gaining the conditioning and strength of a Oia porter is my ideal health goal now. Perhaps time to move to Oia :-).
Sunset in Oia – Byzantine ruins
Oia is at the edge of the island with the Byzantine ruins jutting out at 45 degrees on one side of the island. The Byzantine ruins is the place that you wait up for the sunset.
It is packed at sunset time! Officials close access to this part of the island about an hour before the sunset.
I was told by a national geographic photographer that he reached 3 hours before the sunset and then a fight started right before the sunset and he couldn’t capture the pictures.
My recommendation is to go past the ruins and take the steps to go to the port and stop mid-way to get pictures of the sunset. This is where we got the pictures to the sunset.
Some people take the cruise that starts on the other side of the island to catch the sunset. The cruise brings them near Oia port where they hang around like a bunch young punks and then break up as soon as the sunset is over. I heard that the boat ride is choppy. We unfortunately had a couple of days of greyed out skies and didn’t quite get a great picture. That said, it is surreal to hear whole bunch of tourists clap for the sunset. I couldn’t but help think that most people wouldn’t care two hoots for sunsets whichever city that they come from and here in Oia, we had them clapping for a no-show sunset.
View from Byzantine Ruins – Windmills and Imerovigli
The view from the ruins is gorgeous. You see the famous windmills on one side and the town of Imerovigli on the other. I really loved walking from the ruins to the windmills (a 15 minute low cardio work out :-)).
Imergovigli town is mid-way between Fira and Oia and pretty non descript but with an easy access to Fira bells and some good food.
Staying in Oia and Imerovgili
We stayed in Oia Mare in Oia which is near the base of the windmills and overlooks the ruins. The hotel has rooms which are like the traditional caves of Oia is fantastic, the people managing the hotels are fantastic, the views are fantastic too.
I will highly highly recommend staying here.
We stayed in Senses Boutique hotel in Imergovigli, which has a fantastic view of the Calderra. Unfortunately for us, my wife got allergies to something in the hotel. The hotel staff changed our rooms and things got better but not that much.
I captured some great sunset pictures on the way to Fira from Imerovigli. The three bells of Fira is the best place to capture the Santorini sunset – better than Oia.
Other parts of the Island – Lighthouse, Red Beach and Perissa Beach
The lighthouse and red beach are an interesting touch point but you wouldn’t miss much if you didn’t head there. The next picture is a stop over point just before the lighthouse. The lighthouse itself is unimpressive.
This is where we truly felt that we had hit the island life :-). We headed to a dive restaurant called Tranquillo and truly felt tranquil – perhaps it was the beer :-).
The drive to the other side of the island has numerous bakeries on the way. At one spot, we saw a group of painters and the funny thing was that all the paint marks on their clothes were Oia white because there are no other colors used.
No colors – a terrible life as a painter!
Summary of Santorini trip
So I was on a 2 week vacation — my first in 15 years! Thoroughly enjoyed visiting new places and reading a lot — book notes to come in the next few weeks. Meanwhile, I decided to take it easy on the blog writing for those weeks and I will get back to it starting this week. Meanwhile, here is what is worth sharing this week.
- [Travel] Day 1 in Santorini Fira, Day 2 in Santorini — Walk to Three Bells of Fira
- [Productivity] An article that recommends you to split your day into a makers and a managers schedule
The first day in Fira definitely got into the island mode after a gorgeous sunset and dinner overlooking the Caldera.
We decided to head to the Three Bells of Fira which is part of the iconic church that is often used in pictures.
This walk is beautiful and starts through the shops of Fira and starts on a gradual slope upwards and soon the slope becomes steeper. There aren’t too many directions and everybody is generally walking up the slope in the direction of a different Fira church that comes in practically the first 10 minutes.
On the back side of the church there is a boutique shop that is supposedly very popular called “The cave” – we stopped here for about 10 minutes and soaked in the atmosphere. There are directions on the church wall to head to the 3 bells of Fira.
We quickly moved on. The rest of the walk is on the edge of the cliffs with gorgeous views of the Caldera and the villages Firastefani and Imerovigli.
There were lots of tourists and lots of nice places to take pictures on the way to the bells.
The total round trip for the walk is about 2-3 hours, you can do it much faster but why would you :-)?
Heading to Oia
After the trip, we were all stoked up to head to Oia. The drive from Fira to Oia is about 45 minutes and goes through the mountains and all going up. There is very limited parking in Oia and after some back and forth with the hotel, we finally found the Post office where we parked our car. The post office is the meeting point where the porters from the hotel pick you up and take you to the hotel. I will skip the walk to the hotel and the hotel itself for the next blog.
We spent most the afternoon cooped in the hotel room because of the bright sun and heat. We did walk around Oia for about an hour or so. It is a very small town and about an hour and you get the end to end picture of the town.
The Oia sunset experience is why people head to Oia and the best spot is on the top of Byzantine ruins. What they don’t tell you is the amount of people on the top of the ruins and people start queuing about 3 hours before the sunset — which in the heat of Oia is a bit crazy.
I skipped the ruins because I was standing on the top of our hotel roof which is part of the charming view from the ruins — so I definitely missed the view on the first day. However, we ended up making good friends with a few people from around the world. The topic starter was the crazy traffic on the ruins :-).
One of the things that Santorini is known for, is, donkeys. You cannot really miss them on t-shirts, refrigerator magnets and stuffed toys that show these cute loveable creatures smiling away and living the island life.
A thing to do in Fira is to take a donkey to climb up the 588 steps to the old ferry port.
I was somewhat unhappy to see activity because I support a donkey charity (yes that’s a thing and read the inspiring story of Jean and Bob behind this charity) and find the act of putting a human on a donkeys back inhumane.
I have learnt while supporting Asswin that the general perception that donkeys can really take any absurd amount of weight is wrong and most donkeys end up suffering a broken back. Once an animal has a broken back, they are discarded to die. Note: this is the truth in India and I have no data to say that this happens in Santorini and to an untrained eye(read mine), the donkeys indeed looked healthy.
The second issue is that donkeys aren’t sexy enough or worse yet, they are the butt of jokes which means that there is no charity (except Jean & Bob) who even try to do the work of rescuing and rehabilitating these creatures. This work is really done to help who aren’t helped by anyone else.
But back to the story.
I was out and about for my morning Santorini pictures and I saw a man tying garbage bags in front of the restaurant I had been to previous night. This is where I began to think about garbage disposal in Santorini. The steps are steep, way too steep for humans to pick it up and garbage trucks might not make it to all places. So how do they do it?
Coming from Silicon Valley, I was trying to figure the technological solution to the problem.
Whilst I was trying to figure the puzzle out — I saw a couple of guys bringing their donkeys along. This was 5:30 am in the morning and there was no one on the roads except for me and a few stragglers heading back to their rooms after a night out.
The donkey handler stop in front of the restaurant I was in and start loading the donkey. 1 bag… 2 bags…3 bags — he kept going on and on.
I didn’t keep a count because I was busy watching the face of the donkey.
This donkey was about 2 feet away from me and the face was one of utter hopelessness. The face became further grim with every additional bag that landed on its back and somewhere along a tear streamed down its eye :-(.
Once fully loaded, it had some difficulty finding its footing on the steep steps and along it went on the steps of Fira.
I don’t quite fault the handlers who are busy earning their livelihood — this is so much better than the horses of Iceland where they end up on the dinner table.
But the question is…
At what point will humanity evolve where animal rights equal to human rights. At what point will humanity evolve such that animals aren’t seen as a commodity to be cut, sliced, diced and milked.
What needs to happen to get us to a point where we are even ready to have a conversation that animals need to be treated humanely and I mean treated and not killed humanely.
Seeing the donkeys in Santorini was heart breaking but what made the heart break brutal was to realise that all I could do is write a blog about this fully well knowning that not a whole lot of people will even bother reading it.
(originally published on blueorangeart.com)
This blog is part of series of 8 days in Greece specifically Cyclades islands of Santorini and Mykonos.
Fira in Santorini is the “it” town and this is where the tourists head to. Our driver dropped us off near the main square after picking us up at the airport and I was taken aback to see the number of tourists in the main square.
It seemed like Goa!
We dropped our luggage in the hotel which was a minute away, got some dinner and walked around the Fira streets which are charming.
What to expect in Fira
Expect beautiful charming streets, full of tourists and shops. Lots and lots of restaurants, bars and clubs. Clubs are not my scene so I will skip commenting except to say that I saw tourists girls walking on the street at 5:30 am in the morning after a night of partying. Oh yes another one — the first night here, we could hear the bass from the club next doors upto 3 am!
That aside, the town is charming — to say the least.
Day 1 in Fira – Things to do
Old Ferry Port
Anyways, the walk to the old ferry port is called the 588 steps to the port. It is quite a steep walk down with plenty of opportunities to click pictures of the caldera.
Turns out that donkeys are quite the thing in Santorini and this path had tourists coming up on these donkeys. I support a donkey charity in India and had an unhappy look watching the donkeys lugging humans up the stairs.
The walk down is strenuous and the end reward was a beer. Well not quite – I ordered an Ouzo (dry anise flavoured aperitif) thinking it was a beer and was pleasantly surprised.
Total time – 4 hours
We rented a car and headed down to Kamari beach. Alex (our car rental guy) told us that don’t trust Google and just follow the directions on the road. It was surprising to see how much we have come to depend on Google maps that it was difficult to adjust to signs on the road. That said, it wasn’t too difficult finding our way to Kamari beach — after all it was less than 5 miles away :-). The walk around Kamari definitely reminded me of Goa. Tourist lined shops and restaurants right next to the beach.
The beach doesn’t have sand but black rounded rocks.
We grabbed lunch at an eatery and spent time feeding some lovely cats. My wife is afraid of cats and on this trip she has started becoming friends with some. We then drove back to Fira.
We walked around stores and my wife decided to get her hair done at Stellas while I decided to get a fish manicure. If you haven’t done it before I recommend it.
It’s mind-bending to see that rather than you eating a fish, the fish eats you :-).
Evening sunset and dinner in Fira
As the evening wore on, we headed back up the steps and to watch the views of the Caldera. It was a foggy evening and we didn’t quite have the sunset but the view itself was worth it.
My personal highlight in Fira
While waiting for the sunset, I happen to meet this gentlemen who clearly was 80+ and selling post cards to make a living :-(. He could barely walk and I saw him make his way towards me but I wasn’t quite sure if he was going for the view or coming in for me. After taking about 3 minutes to cover a 10 feet distance, he uttered the word “postcard” and I bought one.
I saw him sit down tired with evidently more work to do for the evening. We approached him and asked if we could buy his entire post card collection which was less than 20Euros and he was elated. He was so happy to do a picture with me.
It’s heartbreaking to see that he has to hustle so late in life — life’s not fair. At the same time, I noticed how incredibly blessed and happy I felt to just bring a smile on his face. Note to self – need to do more to help old people and animals!
Airport to Hotel
We got our hotel to arrange for a pickup from the airport. The charge was 25 EU for 2 person. This was a mini-bus with other passengers along and the ride to Fira was about 25 minutes. The driver insisted on taking payment after dropping us which is not what the hotel had told us. The Hotel person (Elaine) was surprised himself and was going to check back.
Accommodation and Car Rental in Fira
We are staying at the Hotel Tataki (booked from booking.com). The hotel staff specifically Elaine (from Albania) has been really good. This is as central in Fira as you can get – the Caldera is about 2 minutes walk up, the cable car is about 5-7 minutes away, the main square is 2 minutes walk down.
You step out from the hotel and you are in the middle of a bustling tourist street.
They helped us organise car rental with …. (to be updated). Alex who rented the car out was absolutely delightful to deal with. He sat us down, oriented us to Santorini, things to drive around and see. The car itself – a hatchback was $47 Euro/day with complete insurance coverage.
I would highly recommend both the hotel and the car rental company.
(originally posted on blueorangeart.com)