AIBench: A Datacenter AI Benchmark Suite, BenchCouncil


Download User Manual

AIBench User Manual [AIBench-UserManual]

AIBench Download

Please cite our technical report when you publish results that you have obtained with AIBench:

AIBench: An Industry Standard Internet Service AI Benchmark Suite. [Paper, Bibtex]
Wanling Gao, Fei Tang, Lei Wang, Jianfeng Zhan, Chunxin Lan, Chunjie Luo, Yunyou Huang, Chen Zheng, Jiahui Dai, Zheng Cao, Daoyi Zheng, Haoning Tang, Kunlin Zhan, Biao Wang, Defei Kong, Tong Wu, Minghe Yu, Chongkang Tan, Huan Li, Xinhui Tian, Yatao Li, Gang Lu, Junchao Shao, Zhenyu Wang, Xiaoyu Wang, and Hainan Ye. Technical Report 2019. arXiv preprint arXiv:1908.08998.

AIBench: Towards Scalable and Comprehensive Datacenter AI Benchmarking. [Paper, Bibtex]
Wanling Gao, Chunjie Luo, Lei Wang, Xingwang Xiong, Jianan Chen, Tianshu Hao, Zihan Jiang, Fanda Fan, Mengjia Du, Yunyou Huang, Fan Zhang, Xu Wen, Chen Zheng, Xiwen He, Jiahui Dai, Hainan Ye, Zheng Cao, Zhen Jia, Kent Zhan, Haoning Tang, Daoyi Zheng, Biwei Xie, Wei Li, Xiaoyu Wang, and Jianfeng Zhan. 2018 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing (Bench18).

Full software packages of different implementations are available from the following links.

Download AIBench from BenchHub:

Download Datasets

Table 1: The Summary of Data Sets

data sets data size
1 CIFAR-10 60000 color images with the dimension of 32*32
2 ImageNet (1GB,10GB) ILSVRC2014 DET image dataset(unstructured image)
3 LSUN One million labelled images, classified into 10 scene categories and 20 object categories
4 MNIST handwritten digits database which has 60,000
training examples and 10,000 test examples(unstructured image)
5 MovieLens Dataset User’s score data for movies, which has 9,518,231
training examples and 386,835 test examples(semi-structured text)
6 WMT English-German WMT English-German translation dataset
7 MS COCO2014 A large-scale object detection, segmentation, and captioning dataset
8 Cityscapes A new large-scale dataset that contains a diverse set of stereo video sequences recorded in street scenes from 50 different cities
9 LibriSpeech A corpus of approximately 1000 hours of 16kHz read English speech
10 VGGFace2 A large-scale face recognition dataset
11 LFW dataset A database of face photographs
12 Robot pushing dataset 59,000 examples of robot pushing motions
13 ShapeNet Dataset A richly-annotated, large-scale dataset of 3D shapes
14 Gigaword Dataset Nearly ten million documents of the original English Gigaword Fifth Edition
15 Intellifusion data set 77,715 samples from 253 face IDs provided by Intellifusion

Download software

Download prerequisite software packages before using AIBench. Please refer to the user manual. The following packages should be installed firstly, and the running platform is Linux.

Software Version Download
TensorFlow 1.12
PyTorch 1.0.1
GCC 4.8.2