JA1. Introduction¶
Statement¶
Your learning journal entry must be a reflective statement that considers the following questions:
1. Describe what you did¶
This was the second week of this course; it was all about various ways of storing and managing data preparing for data analysis, and the text went through some of the tools that are used in data analysis. I started the week as usual by trying the self-quiz then I watched the lecture videos and started to read the text; unfortunately the text was so long and I had something to read from the previous week so I could not finish my reading fully.
2. Describe your reactions to what you did¶
Apart from the fact that there are lots of text in the reading assignment; I found everything is interesting some of the storing topics are already familiar from previous courses but I still need time to remember and digest the details.
3. Describe any feedback you received or any specific interactions you had. Discuss how they were helpful¶
I did not receive any feedback that is worth mentioning.
4. Describe your feelings and attitudes¶
The difference between storage types, SQL, NoSQL, traditional, analytics, big data…etc are one of my favorite topics in computer science, I am always interested in learning more about them and knowing the deep differences between them; especially when my work changed from using SQL to using DynamoDB, I’m still searching about the motivation behind this change as I’m not convinced that it brought any benefit to the company.
5. Describe what you learned¶
I learned about the different types of storage and the difference between them. I learned about Traditional, transactional, and analytics databases and the differences between them; and how SQL/NoSQL are used in each of them. I learned the strengths and weaknesses of each type of storage and how to choose the right one for the right job.
I also learned about Hadoop and its ecosystem as a way to handle big data by distributing the data and the processing on a cluster of multiple machines. I also learned about R; and how to put all Storage, Hadoop, and R together to create a data analysis pipeline.
6. What surprised me or caused me to wonder?¶
Hadoop was an interesting tool to learn about, just thinking about the problem it solves and its features is interesting; distributing processing, memory and storage across multiple machine is no easy task and it requires precise software that is able to schedule and manage all of this and it is fault tolerant and scalable.
7. What happened that felt particularly challenging? Why was it challenging to me?¶
The written assignment was challenging as the requirements were not super clear; however, I solved it according to my understanding of the requirements; but it is also subjective to my classmates’ point of view when it is time for grading. My point of view that I talked -in details- about various databases, their differences and use cases, then I picked one of theses databases and constructed a pipeline that uses it as data store, along with R, and Hadoop.
8. What skills and knowledge do I recognize that I am gaining?¶
I am gaining knowledge about the different types of databases and their use cases, Hadoop and its ecosystem, R and its ecosystem, and how to put all of these together to create a data analysis pipeline.
9. What am I realizing about myself as a learner?¶
I am realizing that I am a slow reader or the reading assignment of this week and the week before was too long that I couldn’t even touch the surface with the available time to the course.
10. In what ways am I able to apply the ideas and concepts gained to my own experience?¶
We recently changed our database from SQL to DynamoDB, I still believe that a deep understanding of NoSQL databases will greatly help my in the task of modeling entities or creating set of functions that are used to access the data.
11. Describe one important thing that you are thinking about in relation to the activity¶
I am thinking about the utility libraries shipped with a project such as Hadoop and how they are actually build to perform tha tasks required from them.