notes

Big Data and its application?

I & my girlfriend was coming back to our hometown, Bien Hoa city. As usual, every time I was in motion, I always have some really deep thought.

When she & I was discussing about how we could invest money by buying gold and then selling again to make some profit, a thought came to me that we can develop a algorithm that can analyze the up & down of gold market and predict when should buy and when we should sell gold to make the best profit. And this can apply to every thing, for example, real estate, clothing, stock, etc. Or we can gather data from Google Map and then apply principle of Feng Shui to detect and suggest which land we can invest and make profit.

The problem is, how can I develop such system when I don’t have any knowledge and experience in Data Analysis and Big Data algorithm. I think I need to consult with someone else about this.

Uncategorized

Knowing Your Code’s Structure

I haven’t paid much attention to how the classes in my projects communicate with each other. Looking at the package structure, they all look good and well-organized. Then one day, I came across the article Your Code Structure and it made me to seriously think about re-validate my source code.

In the article, Edmund Kirwan used his tool, named Spoiklin Soice, to analyze the modules of JUnit from the early version to the latest one. The result was that as JUnit evolving, its classes had become more tangled and its structure had become a real mess (according to the graph).

This was really interesting because JUnit had been a widely known & used by almost all Java developers and it was developed by the lengedary Kent Beck. In the article, the latest version of JUnit, 4.11, had become really complex due to new features added into the library. I had always thought of JUnit as a powerful and well-tested tool and never paid attention to its structure and design ever since I used it. It could be the case that as time goes by, JUnit would become more and more difficult to extend and mantain due to its complexity.

Well, that was for JUnit, how about my projects? Spoiklin Soice is really interesting tool, I will use it to analyze them to see whether the structure is still controllable.

But what if we knew that the structure of a project had become too complex, what we could do to make it right again? Or there are any principle that we could follow and then keep everything in its right position?

Download and read more about Edmund’s work at his blog: http://edmundkirwan.com/general/spoiklin.html

Uncategorized

Prepare to Getting Real

I’m bored tonight. So I go to my bookmark bar to see whether there is something I can read. Bingo! I came across the @freebook folder and click on Getting Real. This is a free version of 37signal’s book, Getting Real where it talk about how to quickly and efficiently start up a new software product which solve the problem from the begining. Start small, focus and efficient. I will continue to read it tomorrow.

Here is the link to the free book: http://gettingreal.37signals.com

Uncategorized

Agile toolkits

My current company is using Scrum to develop softwares. To me, it is a set of practices and tools which enhances communication, collaboration, trusts, etc between members of a team.
What people are still debating is that, which one, Scrum or Lean is better. I said, why not both? To me, the term Agile has been broaderned to bear many meanings. It points to a lot of processes. But under the hood. It is still 4 statements and 12 principles of Agile Manifesto. Everything else is just a technique. We can decompose Scrum or Lean into techniques and combines several ones to form a new method. To see if it works, we have to try it. But in the end, they are just tools.

Uncategorized

Why there is no SoftHashMap?

The above question bumped into me when I was developing a simple cache using java.util.Map. In the search for the answer, I came across a thread on JavaRanch that had the same question as mine. And there was an answer which had widen my knowledge in soft/weark references in Java.

I quoted the answer here in case the link is inaccessible.

In the few cases, where I used weak references (or soft references), I used the classes, (and queues) directly. Never used the WeakHashMap class. So, never thought about the possibility of the SoftHashMap class. IMO, the WeakHashMap class was written for internal (as support) reasons, and then exposed because others may need it. Maybe that is why SoftHashMap was never written.

In thinking about it a bit more, I don’t think that it is possible to write a SoftHashMap class. Remember that a SoftHashMap isn’t just a WeakHashMap that is using SoftReferences. It has to also not “distrurb” those soft references. Let me explain….

To implement a WeakHashMap, just use WeakReferences on the keys. This means, of course, that you need to do an additional deference to add/remove the key/value pairs. You will also need to attach a ReferenceQueue so that you can detect that a key has been GC’ed — which the WeakHashMap will use to automatically delete the key-value pair.

There is a minor flaw though — particularly if more than one key lands in a bucket. The WeakHashMap needs to deference the key (ie. make it a hard reference temporarily) so that it can use the hashCode() and equals() methods for the operation. This means, that in theory, it is possible for a key (that isn’t referenced by the application) to survive a GC cycle — if by chance theWeakHashMap is doing this operation….. This is probably not a big deal, as the chances are ridiculously low, and it can be picked up on the next GC cycle.

This becomes a major flaw with the SoftHashMap. Remember that SoftReference objects may or may not be GC’ed even if there are no hard references. It is at the whim of the JVM, which decides based on the amount of memory that it needs. The idea is, if there are lots of memory being freed, there is no reason to completely delete a cache that could be useful to the application.

The issue is, the SoftReference determines which instance to GC based on the access — it keeps timestamps of which object has been used and how recent (Least Recently Used). This means, that the temporary conversion of the soft reference to a hard reference (to do the equals()) will affect the LRU. A key that hasn’t been accessed by the application in a very long time, may survive a GC purely because it shares a bucket with a more commonly used key. And unlike the WeakHashMap issue, the chances are not low, and it may not be picked up in the next GC cycle.

Basically, you have a SoftHashMap that has soft references, but isn’t really a soft hash map.

Henry Wong

The link: http://www.coderanch.com/t/487535/Performance/java/WeakHashMap-SoftHashMap

Uncategorized

My father is a …

My father is a writer. A good one. His name is Marc Norman.

He wrote Shakespeare In Love and a hundred other scripts and novels and has two oscars on his mantle.

He told me this about writing…

It’s a full time job. To be successful you must be disciplined. You do it from nine to five, five days a week. Focus on your work for forty hours and put it away at night and on the weekends. When you’re blocked write something else, sonnets, poems, the “other novel”, love letters. The point is you must write full-time. A writer’s talent is a muscle that must be worked out, and like a bicep it will strengthen with use.

Writers write alone, in quiet. Writers don’t write in coffee-shops. Silence is the blank canvas onto which the world of the work is drawn.

I found this on an anwers on Quora: http://www.quora.com/Writing/What-should-everyone-know-about-writing

Uncategorized

SQL isn’t dead. JooQ is a new fresh thing

After working with 3 layers, DAO, JPA and Hibernate and stuff, I think writing native SQL code is going to disappear. However, til today, I just found a new framework that support you to write SQL command by Java code. Their fluent API is so good that it seems identical with real SQL command.

Check out JOOQ – Java Object Oriented Querying;

http://www.jooq.org/

P.s: Guess what! It’s type-safe.

today-i-found

Little things matter

Today, I’ve read a entry in Quora that actually Made Me Thing (MMT). It was a survey “What is the nicest thing anyone has ever done to you?” and there was a man writing that some woman had helped him back when he was travelling in Swizertland and making his way to his home in Rome. The woman had spent over 2 hours to sit with him, hopping trains in order to make sure he made it way home safe. He had been moved by the action and really appreciated the woman’s action. He even stated that to him the most nicest country is Swizertland.

After reading the post, I just realized that doing a little thing with your heart really matters. You don’t need to do big things to impress people. Doing a little thing with your heart in it and specially with careness is actually the most thing that help you “win” people.

So, how about my closet person? My girlfriend? My mother? My sister? My colleagues? What litte things I can do for them to make them feel better NOW?

journal

bookmarking + mind mapping

Recently, I’ve been using an interesting and useful free online service, GetPocket which helps me to bookmark articles, papers, webpages, etc to read it later. The service even extracts the content and displays it in elegant format. It even allows me to tag, search, add favorite, etc. In short, it’s cool.

I’ve been bookmarking a lot of good articles and links that I’m interested and plan to read them on my free time. Until today I’ve suddenly realized that I actually need a similar service as GetPocket but allows the user to chain/connect them as a MindMap.

Usually, one of my hobby is that for certain topic/keyword, I start searching materials about them. For each article I reach, I read a few first paragraphs and if I encounter a link to another one, I immediately open the link. The routine is as follow:

1 – open
2 – read a few paragraphs until encounter (interesting) links/keywords/concepts
3 – open them
4 – go to 1 repeat.
5 – if there is nothing new, I come back to the previous articles.

It looks like I’m digging up a articles tree, or rather travelling a material map. Then an idea popped up in my mind: Why not combining the two: bookmarking the articles and mind mapping.

I want every time I bookmark a link, I can tag it and link the article to a mind map with some keyword and some description for the article. Then by the time I read back the articles, I can follow the graph and regain the knowledge. The application could render the articles and its graph to follow, just as a mindmap.

Uncategorized

Bugs lie in uncertain code

Today after having lunch, I went around articles queue in order to read one or some. As reading the Architecture-Breaking Bugs, a lot of articles I had read before came back to my mind.

The content of article was about flaws in the architecture which the engineers often overlooked, turned out disasters. There was a link to another article which told the story of Airline 5 and has it crashed because only 1 bug at 1 line of code. Going through the articles made me remind of “Clean code” of Uncle Bob, “Don’t Assume It, Prove It” of The Pragmactic Programmer.

Why? Because at the time I was reading the article, I had finished basically a feature in my current project, Migration Process. The process is for migrating old workflow cases into new Process Model Version to avoid too many versions running in parallel. My group was barely finished the process on Friday and had to show it to the Product Owner on next Monday. Honestly, the process was untested, and I was sitting there doing nothing. There were a lot of things to be tested, a lot of scenarios to tests, but I chose to test several of them under the assumption the process will work. HOPE!.

God damn it! I had to go back to do something about it.