This tutorial is aimed at presenting BigDataBench 5.0---a scalable big data and data-driven artificial intelligence (in short, AI) benchmark suite. We are glad to introduce the following interesting topics:

(1) The challenges and motivation for characterizing modern Big Data and AI workloads.

(2) Why scalable benchmarking methodology matters? What is a data motif? How many data motifs can characterize and compose modern comprehensive Big Data and AI workloads? Micro benchmarks of Modern Big Data and AI workloads.

(3) The essentials of modern big data and AI workloads, e.g., Image classification, Image generation, Text-to-Text Translation, Image-to-text, Image-to-image, Speech-to-text, Word embedding, Face embedding, Object detection, Recommendation, Graph Model, clustering, classification, PageRank, Feature extraction, Search engine indexing. Their data sets, algorithms, and involved data motifs.

(4) Two end-to-end application benchmarks: Mixed data center workloads, and E-commerce search (Joint work with Alibaba).

(5) The proxy benchmarks for modern Big Data and AI workloads that shorten the execution time by 100s times.

(6) How to use BigDataBench 5.0?

(7) How to contribute to BigDataBench? How to join BenchCouncil (http://www.benchcouncil.org).

We will give a tutorial on BigDataBench at HPCA 2019 in Washington D.C., USA.

February 16, 2019 (Saturday),08:30 - 17:10 (Full Day)

ROOM:University of DC

As a multi-discipline, i.e., system, architecture, data management and machine learning, research and engineering effort from both industry and academia, BigDataBench (IISWC' 13, HPCA' 14, PACT' 16, TPDS' 17, PACT' 18, IISWC' 18) is a scalable big data and AI benchmark suite. It is roughly estimated that there are 600+ published paper citing or using BigDataBench since 2014.

We consider each big data and AI workload as a pipeline of one or more classes of units of computation performed on different initial or intermediate data inputs, each of which we call a data motif. For the first time, among a wide variety of big data and AI workloads, we identify eight data motifs— including Matrix, Sampling, Logic, Transform, Set, Graph, Sort and Statistic computation, each of which captures the common requirements of each class of unit of computation while being reasonably divorced from individual implementations. Significantly different from the traditional kernels, a data motif’s behaviors are affected by the sizes, patterns, types, and sources of different data inputs; Moreover, it reflects not only computation patterns, memory access patterns, but also disk and network I/O patterns.

The current version — BigDataBench 5.0 is significant upgrade, and provides comprehensive Big Data and AI benchmark suite, including online services, offline analytics, graph analytics, AI, data warehouse, NoSQL, and streaming workloads. Our benchmark suite includes micro benchmarks, each of which is a single data motif, components benchmarks, which consists of the data motif combinations, and end-to-end application benchmarks, which are the combinations of component benchmarks. Also, we propose using the combinations of the eight data motifs with different weights to mimic the benchmarks in BigDataBench. Our proxy benchmarks shorten the execution time by 100s times on the real systems and they are qualified for both earlier architecture design and later system evaluation across different architectures.