Abstract
The Advance of web-based technologies have brought radical changes to web site design and web service usage, primarily in terms of interactive contents and user engagement in collaboration and information sharing. In nutshell the web has been transformed from static media to the preferred communication media where the user is a key player in the creation of his experiences. The increase in the popularity of social networks on the Web has shaken up traditional models in different areas, including learning. Many individuals have resorted to social networking to educate themselves. Such learning is close to natural learning, the learner is autonomous to draw the pathway which best suits his individual needs in order to upgrade his skills. Several training organizations use the Twitter platform to announce the training they provide. We conduct an experiment on twitters data which are related to the training themes in Big Data and Data Science, we perform an exploratory analysis and extract the top group of connected hashtags using the Graph X library provided by the Spark framework. Data that come from the Twitter platform is produced at high speed and in a complex structure. This leads us to use a distributed infrastructure based on two efficient frameworks Apache Hadoop and Spark. Data ingestion layer is built by combining two frameworks Apache Flume and Kafka.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cameron, R., Harrison, J.L.: The interrelatedness of formal, non-formal and informal learning: evidence from labour market program participants. Aust. J. Adult Learn. 52(2), 277–309 (2012)
McPherson, M., Budge, K., Lemon, N.: New practices in doing academic development: Twitter as an informal learning space. Int. J. Acad. Dev. 20(2), 126–136 (2015)
Wadhwa, P., Bhatia, M.P.S.: Social networks analysis: trends, techniques and future prospects. In: Fourth International Conference on Advances in Recent Technologies in Communication and Computing (ARTCom 2012), Bangalore, India, pp. 1–6 (2012)
White, T.: Hadoop: The Definitive Guide. O’Reilly Media, Inc., Newton (2012)
Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing (2010)
Ghemawat, S., et al.: The Google File System. ACM SIGOPS Operating Systems Review (2013)
Zaharia, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of 9th USENIX Conference on Networked Systems Design and Implementation, p. 2 (2012)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Bonchi, F., Castillo, C., Gionis, A., Jaimes, A.: Social network analysis and mining for business applications. ACM Trans. Intell. Syst. Technol. (TIST) Arch. 2(3), 37 (2011). Article 22
Kim, Y., Hwang, E., Rho, S.: Twitter news-in-education platform for social collaborative and flipped learning. J. Supercomput. Springer, 1–19 (2016). https://doi.org/10.1007/s11227-016-1776-x
Aramo-Immonen, H., Kärkkäinen, H., Jussila, J.J., Joel-Edgar, S., Huhtamäki, J.: Visualizing informal learning behavior from conference participants’ Twitter data with the Ostinato model. J. Comput. Hum. Behav. Arch. 55(PA), 584–595 (2016)
Huhtamäki, J., Russell, M.G., Rubens, N., Still, K.: Ostinato: the exploration-automation cycle of user-centric, process-automated data-driven visual network analytics. In: Matei, S., Russell, M., Bertino, E. (eds.) Transparency in Social Media, pp. 197–222. Cham, Computational Social Sciences, Springer (2015). https://doi.org/10.1007/978-3-319-18552-1_11
Chen, X., Vorvoreanu, M., Madhavan, K.: Mining social media data for understanding students’ learning experiences. IEEE Trans. Learn. Technol. 7(3), 246–259 (2014)
Chandra, S., Khan, L., Muhaya, F.B.: Estimating Twitter user location using social interactions–a content based approach. In: IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, Boston, MA, pp. 838–843 (2011)
Twitter libraries homepage. https://developer.twitter.com/en/docs/developer-utilities/twitter-libraries. Accessed 24 Feb 2018
Shreedharan, H.: Using Flume. O’Reilly Media, Inc., Sebastopol (2014)
Vohra, D.: Apache kafka. In: Practical Hadoop Ecosystem. Apress, Berkeley, CA Apache (2016)
Apache Flume homepage. https://flume.apache.org/. Accessed 24 Feb 2018
Bondy, J.A., Murty, U.S.R.: Graph Theory with Applications. American Elsevier Publishing Company, New York (1976)
MurmurHash3 documentation. https://www.scala-lang.org/files/archive/api/2.11.0-M4/index.html#scala.util.hashing.MurmurHash3$. Accessed 24 Feb 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Chaffai, A., Hassouni, L., Anoun, H. (2018). Informal Learning in Twitter: Architecture of Data Analysis Workflow and Extraction of Top Group of Connected Hashtags. In: Tabii, Y., Lazaar, M., Al Achhab, M., Enneya, N. (eds) Big Data, Cloud and Applications. BDCA 2018. Communications in Computer and Information Science, vol 872. Springer, Cham. https://doi.org/10.1007/978-3-319-96292-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-96292-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96291-7
Online ISBN: 978-3-319-96292-4
eBook Packages: Computer ScienceComputer Science (R0)