Skip to content

7. Trends in Big data

Transformational Issues of Big Data and Analytics in Networked Business 1

  • 5Vs of Big Data: Volume, Velocity, Variety, Veracity, and Value.
  • Big data can be transactional or non-transactional; external or internal.
  • Big data sources:
    • Large-scale enterprise systems: systems used by large companies such as ERP (Enterprise Resource Planning), CRM (Customer Relationship Management), SCM (Supply Chain Management), and HRM (Human Resource Management).
    • Online social graphs.
    • Mobile devices.
    • Internet of Things (IoT).
    • Open and public data: weather, traffic, maps, environment, etc.
  • The use of the term business analytics is now becoming standard to communicate the full life cycle of enhanced data-driven business decision-making.
  • Business analytics borrows from statistics, econometrics, machine learning, and distributed computing paradigms.
  • Business analytics differs from data science in that analytics focuses on better data-driven decision-making in an organizational context.

Data Quality

  • Big data is the key ingredient in the estimation of many analytical models and the well-known GIGO (garbage in, garbage out) principle continues to apply.
  • We note that data will never be of perfect quality, but any acceptable margin of error must be clearly identified, including the impacts of accepted error levels on the transformational impact of the resulting analytical models.
  • There exists a large body of literature studying data accuracy, data completeness, data latency, data security, data interpretability, and data traceability.
  • Analytical techniques include regression, decision trees, neural networks, support vector machines, and ensemble methods.
  • Small improvements in analytical performance can result in substantial profit gains or cost savings.
  • Data validation through triangulation, which collects and compares data from multiple sources:
    • Direct market observations, Indirect empirical data, and Focused survey data.
    • Example: compare your internal transaction log with an external source (e.g. bank statement), and with a third-party data provider that conducts its own surveys about your transactions; then contrast and compare the results to increase confidence in the data quality.
    • Aka, cross-validation.
  • Investing in quality data is a long-term, demanding process with high accompanying costs, the investment will be unlikely unless senior management trusts and understands the importance of quality data to the development of high-value-added analytic models.

Methodological Paradigms and Challenges of Big Data

  • Big data allows us to leverage both prediction and causal analysis.
  • Big data research borrows techniques from machine learning, classical statistics, and econometrics. It designs experiments (A/B or multivariate testing) to test existing theories and hypotheses, develop new theories, and create large-scale business value.
  • It tries to conduct experiments and collect the data needed to obtain answers to a variety of questions including:
    • (1) effects of peer influence,
    • (2) impacts of the influence of dynamic ties,
    • (3) impacts of anonymity on online relationships,
    • (4) results from alternative pricing strategies for digital media,
    • (5) the sway of carefully designed next-generation recommender systems,
    • (6) the changing preference structures of Generation Y and Z consumers.

Innovative Big Data Applications

  • Online-to-offline (O2O) commerce:
    • The use of big data to drive online traffic to physical stores.
    • Example: QB House, a Japanese barbershop chain, puts sensors in its chairs to track available seats and wait times, and reflects this information on its website (and mobile app) to direct customers to the nearest available location.
  • Network of smart vehicles:
    • As an example of Internet-of-things, smart vehicles, or Internet-of-cars, are equipped with an onboard diagnostics (OBD) device so that the behaviors of drivers and car components can be monitored.
    • The recorded data is then used by vehicle owners, drivers, vehicle managing companies, and insurance companies to make better decisions using analytics and value-driven data modeling.
    • The results of such applications of big data should include better traffic flow, fairer insurance premiums, and better fleet management.
    • Usage-based insurance (UBI):
      • Insurance premiums are based on the driving behavior of the insured, which is monitored by the OBD device.
      • This is not a new concept, but applying it with big data allowed to be profitable and scalable.
  • Proactive Customer Care:
    • Location-based services offered by mobile communication companies offer travelers detailed information on where to buy things as the need arises.
    • Location-based services using real-time big data and analytics enable companies to understand how their customers’ needs and interests change as the customers move locations.

Disruptive Impacts of Big Data

  • Business Analysts Retooling:
    • Business analysts must be trained in big data capture, big data analysis, big data modeling, and big-data-based decision-making.
    • An approach for retooling business analysts is to create alliances among universities and companies to develop big data academic and professional education programs.
  • Integration of Data and Social Sciences:
    • The availability of micro-level behavioral data creates collaboration opportunities for company researchers, social scientists, and data scientists.
    • Big data makes innovative projects possible and opens opportunities for investigations that can yield deeper insights into and understanding human motivation, consumer choices, social phenomena, and the micro-level impact of business activities.
  • Breakdown of Traditional Business Boundaries:
    • The opportunities provided to companies through big data analytics have begun to break down a variety of traditional business boundaries.
    • E-commerce giants such as Alibaba and Tencent have moved into banking and now offer savings funds and online insurance services, causing uncertainties and major disruptions to China’s banks and multinational banks.
    • The eCommerce companies possess new business features that are not available to traditional banks, including existing relationships with hundreds of millions of eCommerce customers.

Challenges and Research Opportunities of Big Data and Analytics

  • Ubiquitous Informing:
    • Individuals and businesses record what they find interesting, store this information for themselves or others, and share the data for personal and/or business purposes.
    • The challenge is to use this information to create value for individuals and businesses as it is hard to fathom.
  • Implementation Environments:
    • The cost of implementing big data environments is high, and without a deep understanding of tools such as Hadoop and Spark, these tools may stay underutilized.
  • Integration issues:
    • Integration between data of various types and sources is a challenge.
  • Value assessment:
    • Big data is characterized using statistical terms, such as mean, median, etc., which makes it difficult for non-experts to understand the value of the data.
    • Analytic models should be understandable to decision-makers.
  • Regulatory Compliance:
    • Example: Basel III Capital Requirements Accord introduced by the Bank of International Settlements for credit risk modeling.
  • Managing Analytic Decisions:
    • Drawing out decision-ready inferences from big data analytics influences and enhances the firm’s decision-making is hard.
  • Return on Investment and Trust:
    • The return on investment in big data analytics is hard to measure.

Artificial Intelligence and Big Data in Public Health 2

  • Big Data includes information that is structured, semi-structured, or unstructured, and there may be complex interrelationships that are syntactic, semantic, social, cultural, economic, and organizational in nature.
  • There may also be epistemic uncertainties that enter the data processing pipeline and hinder decision-making by humans and computers, including (a) data corruption by noise and artifacts, (b) data entry errors, © duplicate records, (d) missing records, and (e) incomplete digital records relating to information on date, age, gender, or other variables.

A Framework for Identifying and Analyzing Major Issues in Implementing Big Data and Data Analytics in E-Learning 3

Fast Data: Moving Beyond Big Data’s Map-Reduce 4

Fundamentals of MapReduce with MapReduce Example 5

What is MapReduce in Hadoop? Big Data Architecture 6

Marketing Analytics for Data-Rich Environments 7

References

  1. Baesens, B., Bapna, R., Marsden, J. R., Vanthienen, J., & Zhao, J. L. (2016). Transformational issues of big data and analytics in networked business. Management Information Systems Research Center Quarterly, 40(4), 807–818. https://www.jstor.org/stable/26629677 ↩

  2. Benke, K., & Benke, G. (2018). Artificial intelligence and big data in public health. International Journal of Environmental Research and Public Health, 15(12), 2796. https://www.mdpi.com/1660-4601/15/12/2796/htm ↩

  3. Corbeil, M. E, Corbeil, J.R, & Khan, B. H. (2017). A framework for identifying and analyzing major issues in implementing big data and data analytics in e-learning: introduction to special issue on big data and data analytics. Educational Technology 57, (1),3–9. http://www.jstor.org/stable/44430534 ↩

  4. Lev-Libfeld, A., & Margolin, A. (2019). Fast data: Moving beyond big data’s map-reduce. Journal of Geopython, (1), https://arxiv.org/ftp/arxiv/papers/1906/1906.10468.pdf ↩

  5. Sinha, S. (2016, November 15). Fundamentals of MapReduce with MapReduce example. Edureka. https://medium.com/edureka/mapreduce-tutorial-3d9535ddbe7c ↩

  6. Taylor, D. (2022, September 17). What is MapReduce in Hadoop? Big data architecture. Guru99. Retrieved September 27, 2022, from https://www.guru99.com/introduction-to-mapreduce.html ↩

  7. Wedel, M., & Kannan, P. K. (2016). Marketing analytics for data-rich environments. Journal of Marketing, 80(6), 97–121. http://www.jstor.org/stable/44134975 ↩