2020 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing (Bench 2020)
Nov 15-16, 2020, 8:00 am EST
Time (EST) | Event | Video |
---|---|---|
08:00 - 08:10 | Opening Remark (Professor Jianfeng Zhan) | [Slides] |
Session A | ||
08:10 - 09:00 | Chair: Professor Geoffrey Fox, Indiana University Keynote - Award Lecture (BenchCouncil Rising Star Award): Scientific Benchmarking of Parallel Computing Systems Professor Torsten Hoefler, ETH Zurich ![]() Bio: Torsten is a Professor of Computer Science at ETH Zürich, Switzerland. Before joining ETH, he led the performance modeling and simulation efforts of parallel petascale applications for the NSF-funded Blue Waters project at NCSA/UIUC. He is also a key member of the Message Passing Interface (MPI) Forum where he chairs the "Collective Operations and Topologies" working group. Torsten won best paper awards at the ACM/IEEE Supercomputing Conference 2010 (SC10), EuroMPI 2013, SC13, SC14, SC19, IPDPS'15, ACM HPDC'15 and HPDC'16, ACM OOPSLA'16, and other conferences. He published numerous peer-reviewed scientific conference and journal articles and authored chapters of the MPI-2.2 and MPI-3.0 standards. For his work, Torsten received the ACM Gordon Bell Prize in 2019, the IEEE TCSC Award of Excellence (MCR) in 2019, ETH Zurich's Latsis Prize in 2015, the SIAM SIAG/Supercomputing Junior Scientist Prize in 2012, and the IEEE TCSC Young Achievers in Scalable Computing Award in 2013. Following his Ph.D., he received the Young Alumni Award 2014 from Indiana University. Torsten was elected into the first steering committee of ACM's SIGHPC in 2013 and he was re-elected in 2016. He was the first European to receive many of those honors. His research interests revolve around the central topic of "Performance-centric System Design" and include scalable networks, parallel programming techniques, and performance modeling. Additional information about Torsten can be found on his homepage at htor.inf.ethz.ch. |
[Slides] [Video] |
Best Paper (Chair: Dr. Wanling Gao, Institute of Computing Technology, Chinese Academy of Sciences) | ||
9:00 - 9:15 | Characterizing the Sharing Behavior of Applications using Software Transactional Memory Douglas Pereira Pasqualin (Universidade Federal de Pelotas), Matthias Diener (University of Illinois at Urbana-Champaign), André Rauber Du Bois (Universidade Federal de Pelotas) and Mauricio Lima Pilla (Universidade Federal de Pelotas) |
[Paper] [Video] |
9:15 - 9:30 | swRodinia: A Benchmark Suite for Exploiting Architecture Properties of Sunway Processor Bangduo Chen (Beihang University), Mingzhen Li (Beihang University), Hailong Yang (Beihang University), Zhongzhi Luan (Beihang University), Lin Gan (Tsinghua University), Guangwen Yang (Tsinghua University) and Depei Qian (Beihang University) |
[Paper] [Video] |
9:30 - 9:45 | Break | |
Session B | ||
09:45 - 10:35 | Chair: Professor Lizy Kurian John, University of Texas at Austin Keynote - Award Lecture (BenchCouncil Achievement Award): It’s a Random World: Learning from Mistakes, Errors, and Noise Professor David J. Lilja, Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis (USA) ![]() Bio: David J. Lilja received Ph.D. and M.S. degrees in Electrical Engineering from the University of Illinois at Urbana-Champaign, and a B.S. in Computer Engineering from Iowa State University in Ames. He is currently a Professor of Electrical and Computer Engineering, and is a member of the graduate faculties in Computer Science and Data Science, at the University of Minnesota in Minneapolis. Previously, he served as the head of the ECE department at the University of Minnesota, worked as a research assistant at the Center for Supercomputing Research and Development at the University of Illinois, and was a development engineer at Tandem Computers Incorporated in California. His main research interests include computer architecture, parallel processing, high-performance storage systems, and computer systems performance analysis. He is a Fellow of both the Institute of Electrical and Electronics Engineers (IEEE) and the American Association for the Advancement of Science (AAAS) for contributions to the statistical analysis of computer performance. |
[Slides] [Video] |
Supercomputing (Chair: Dr. Zhen Jia from Amazon) | ||
10:35 - 10:45 | Optimization of the Himeno Benchmark for SX-Aurora TSUBASA Akito Onodera (Tohoku University), Kazuhiko Komatsu (Tohoku University), Soya Fujimoto (NEC Corporation), Yoko Isobe (NEC Corporation), Masayuki Sato (Tohoku University) and Hiroaki Kobayashi (Tohoku University) |
[Paper] [Video] |
Data Management & Storage (I) (Chair: Dr. Zhen Jia from Amazon) | ||
10:45 - 11:00 | Impact of Commodity Networks on Storage Disaggregation with
NVMe-oF Arjun Kashyap, Shashank Gugnani and Xiaoyi Lu (The Ohio State University) |
[Paper] [Video] |
11:00 - 11:15 | K2RDF: A Distributed RDF Data Management System on Kudu and
Impala Xu Chen, Boyu Qiu, Jungang Xu and Renfeng Liu (University of Chinese Academy of Sciences) |
[Paper] [Video] |
11:15 - 11:25 | OStoreBench: Benchmarking Distributed Object Storage Systems
Using Real-world Application Scenarios Guoxin Kang (Institute of Computing Technology Chinese Academy of Sciences), Defei Kong (ByteDance), Lei Wang (Institute of Computing Technology Chinese Academy of Sciences) and Jianfeng Zhan (Institute of Computing Technology Chinese Academy of Sciences) |
[Paper] [Video] |
Time (EST) | Event | Video |
---|---|---|
Session C | ||
08:00 - 08:50 | Chair: Professor Felix Wolf, Department of Computer Science of Technische Universität Darmstadt in Germany AIBench and Its Performance Rankings Professor Jianfeng Zhan, Chair of BenchCouncil Steering Committee ![]() Dr. Jianfeng Zhan founds and chairs BenchCouncil. He served as IEEE TPDS Associate Editor since 2018. He received the second-class Chinese National Technology Promotion Prize in 2006, the Distinguished Achievement Award of the Chinese Academy of Sciences in 2005, and IISWC Best paper award in 2013, respectively. Jianfeng Zhan received his B.E. in Civil Engineering and MSc in Solid Mechanics from Southwest Jiaotong University in 1996, and 1999, and his Ph.D. in Computer Science from Institute of Software, CAS and UCAS in 2002. |
[Slides] [Video] |
Data Management & Storage (II) (Chair: Professor Zhibin Yu, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences) | ||
8:50 - 9:05 | Artemis: An Automatic Test Suite Generator for Large Scale OLAP Database Kaiming Mi, Chunxi Zhang, Weining Qian and Rong Zhang (East China Normal University) |
[Paper] [Video] |
9:05 - 9:15 | ConfAdvisor: An Automatic Configuration Tuning Framework for NoSQL Database Services with a Black-box Approach Pengfei Chen, Zhaoheng Huo, Xiaoyun Li, Hui Dou and Chu Zhu (Sun Yat-sen University) |
[Paper] [Video] |
Benchmarking on GPU (Chair: Professor Zhibin Yu, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences) | ||
9:15 - 9:30 | Parallel sorted sparse approximate inverse preconditioning algorithm on GPU Chen Qi (Nanjing Normal University), Gao Jiaquan (Nanjing Normal University), Chu Xinyue (Nanjing Normal University) and He Guixia (Zhejiang University of Technology) |
[Paper] [Video] |
9:30 - 9:45 | ComScribe: Identifying Intra-node GPU communication Palwisha Akhtar, Erhan Tezcan, Fareed Mohammad Qararyah and Didem Unat (Koç University, Turkey) |
[Paper] [Video] |
9:45 - 10:00 | Break | |
Session D (Chair: Professor Felix Wolf, Department of Computer Science of Technische Universität Darmstadt in Germany) | ||
10:00 - 10:50 | Keynote: Benchmarking Quantum Computers Professor Kristel Michielsen, Institute for Advanced Simulation, Jülich Supercomputing Centre ![]() In order to evaluate quantum computing as a new compute technology, profound test models and benchmarks are needed to compare quantum computing and quantum annealing devices with trustworthy simulations on digital supercomputers. These simulations provide essential insight into their operation, enable benchmarking and contribute to their design. We present results of benchmarking quantum computing hardware and software. We show benchmarking outcomes for the IBM Quantum Experience and CAS-Alibaba gate-based quantum computers, the D-Wave quantum annealers D-Wave 2000Q and Advantage, and for the approximate quantum optimization algorithm (QAOA) and quantum annealing. For this purpose, also simulations of both types of quantum computers are performed by first modeling them as zero-temperature quantum systems of interacting spin-1/2 particles and then emulating their dynamics by solving the time-dependent Schrödinger equation. Bio: Prof. Dr. Kristel Michielsen received her PhD from the University of Groningen (the Netherlands) for work on the simulation of strongly correlated electron systems in 1993. Since 2009 she is group leader of the research group Quantum Information Processing at the Jülich Supercomputing Centre, Forschungszentrum Jülich (Germany) and is also Professor of Quantum Information Processing at RWTH Aachen University (Germany). Kristel Michielsen and her research group have ample experience in performing large-scale simulations of quantum systems. She has expertise in, on the one hand simulating quantum computers and quantum annealers, and on the other hand in benchmarking and studying prototype applications for this new compute technology by using the various quantum computing and quantum annealing systems that are nowadays available. Together with Prof. Lippert she is building up the Jülich Universal Infrastructure for Quantum computing (JUNIC) at the Jülich Supercomputing Centre. |
|
Application & Dataset | ||
10:50 - 11:05 | A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection Ning Li, Tao Li, Chunyu Hu, Kai Wang and Hong Kang (Nankai university) |
[Paper] [Video] |
11:05 - 11:20 | MAS3K: An Open Dataset for Marine Animal Segmentation Lin Li (Ocean University of China), Eric Rigall (Ocean University of China), Junyu Dong (Ocean University of China) and Geng Chen (Inception Institute of Artificial Intelligence, United Arab Emirates) |
[Paper] [Video] |
11:20 - 11:30 | Benchmarking Blockchain Interactions in Mobile Edge Cloud Software Systems Hong-Linh Truong (Aalto University) and Filip Rydzi (Independent) |
[Paper] [Video] |
11:30 - 12:00 | Invited Talk: DataBench Toolbox – supporting Big Data and AI Benchmarking Dr. Arne J. Berre (Chief Scientist, SINTEF Digital), Tomás Pariente Lobo (Associate Head of AI, Data & Robotics Unit, Atos), Dr. Todor Ivanov (Senior consultant at Lead Consult) Abstract: The DataBench Toolbox offers support for big data and AI benchmarking based on existing efforts in the benchmarking community. The DataBench framework is based on classification of benchmarks using a generic pipeline structure for Big Data and AI pipelines related to the Big Data Value Association (BDVA) Reference Model and the ISO SC42 AI Framework. Based on existing efforts in big data benchmarking and enabling inclusion of new benchmarks that could arise in the future, the DataBench Toolbox provides an environment to search, select and deploy big data benchmarking tools, giving the possibility to identify technical metrics and, also relate to and derive business KPIs for an organization, in order to support evidence Based Big Data and AI Benchmarking to improve Business Performance The Handbook and the DataBench toolbox are essential components of the DataBench project results. The DataBench Toolbox is a software tool which will provide access to benchmarking services, KPIs and various types of knowledge: the DataBench Handbook plays a complementary role to the Toolbox by providing a comprehensive view of the benchmarks referenced in the Toolbox, of how technical and business benchmarking can be linked. The DataBench Handbook and Toolbox are aimed at industrial users and technology developers who need to make informed decisions on Big Data and AI Technologies investments by optimizing technical and business performance. Bio: ![]() ![]() ![]() |
[Video] [Slides 1] [Slides 2] [Slides 3] |