BenchCouncil 每年将颁发以下奖项及奖金:
BenchCouncil 成就奖(3000 美元)
BenchCouncil 新星奖(1000 美元)
BenchCouncil 优秀博士学位论文奖(1000 美元)
BenchCouncil 最佳论文奖(1000 美元)
BenchCouncil Tony Hey 最佳学生论文奖(1000 美元)
BenchCouncil 可复现研究卓越奖
BenchCouncil 各奖项评选委员会由 BenchCouncil 指导委员会成员及往届获奖者共同组成。
我们诚挚欢迎产业界领军企业作为赞助方,为上述奖项提供奖金支持。
本奖项授予在基准测试、数据、标准、评估及优化领域作出长期杰出贡献的资深人士。获奖者将自动获得 BenchCouncil 会士(Fellow)荣誉称号,并享有 3000 美元奖金。
获奖理由“在工作负载特征描述方法学与工具领域作出了开创性贡献;他是 Sniper 的主要开发者,该模拟器是一款快速、精确且支持并行处理的 x86 多核模拟器。”
代表性论文
有关 Eeckhout 教授的突出贡献详情,敬请参阅此处:https://users.elis.ugent.be/~leeckhou/
[1] Carlson, Trevor E., Wim Heirman, and Lieven Eeckhout. "Sniper: Exploring the level of abstraction for scalable and accurate parallel multi-core simulation." In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1-12. 2011.
[2] Georges, Andy, Dries Buytaert, and Lieven Eeckhout. "Statistically rigorous java performance evaluation." ACM SIGPLAN Notices 42, no. 10 (2007): 57-76.
[3] John, Lizy Kurian, and Lieven Eeckhout, eds. Performance evaluation and benchmarking. CRC Press, 2018.
2023年度BenchCouncil成就奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Jack J. Dongarra, University of Tennessee (Since 2022)
John L. Henning, SPEC CPU Subcommittee and Oracle (Since 2023)
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Retired)
Prof. David J. Lilja, University of Minnesota, Minneapolis (Retired)
Prof. Lizy Kurian John, the University of Texas at Austin (Retired)
获奖理由“他是 SPEC CPU 2000、SPEC CPU 2006 及 SPEC CPU 2017 的主要设计者/核心贡献者。该系列基准测试程序被全球工业界与学术界广泛采用,是 CPU 设计与评估的事实标准。”
代表性论文
有关 JOHN's 的突出贡献详情,敬请参阅此处:https://dl.acm.org/profile/81100057428
[1] John L. Henning. SPEC CPU2006 benchmark descriptions. SIGARCH Comput. Archit. News 34, 4 (September 2006), 1–17.
[2] John L. Henning. SPEC CPU2000: Measuring CPU performance in the new millennium. Computer 33.7 (2000): 28-35.
[3] John L. Henning. A Retrospective Look at SPEC Benchmarking, including Successes and Failures. ModSim 2022. August 10–12, 2022.
2022年度BenchCouncil成就奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin (Retired)
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020) (Retired)
Prof. David J. Lilja, University of Minnesota, Minneapolis (Since 2021) (Retired)
Prof. Jack J. Dongarra, University of Tennessee (Since 2022)
获奖理由“其贡献卓著,主要体现在高质量数学软件的开发、测试、文档化以及高性能计算系统的基准测试标准化进程中。”
代表性论文
[1] Anderson, E., Bai, Z., Bischof, C., Blackford, L. S., Demmel, J., Dongarra, J., ... & Sorensen, D. (1999). LAPACK users' guide. Society for industrial and applied mathematics.
[2] Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J., Dongarra, J., ... & Van der Vorst, H. (1994). Templates for the solution of linear systems: building blocks for iterative methods. Society for Industrial and Applied Mathematics.
[3] Gabriel, E., Fagg, G. E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J. M., ... & Woodall, T. S. (2004). Open MPI: Goals, concept, and design of a next generation MPI implementation. In Recent Advances in Parallel Virtual Machine and Message Passing Interface: 11th European PVM/MPI Users’ Group Meeting Budapest, Hungary, September 19-22, 2004. Proceedings 11 (pp. 97-104). Springer Berlin Heidelberg.
2021年度BenchCouncil成就奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020)
Prof. David J. Lilja, University of Minnesota, Minneapolis (Since 2021)
获奖理由“其学术贡献涵盖三大维度:一是提炼测量、仿真与分析建模的实用方法论;二是创立 MinneSPEC,为仿真驱动的体系结构研究提供关键基准工具;三是揭示软硬件及架构-电路的交互机理,为系统性能突破开辟新径。”
代表性论文
[1] David J Lilja. (2005). Measuring computer performance: a practitioner's guide. Cambridge university press.
[2] AJ KleinOsowski, David J Lilja. (2002). MinneSPEC: A new SPEC benchmark workload for simulation-based computer architecture research. IEEE Computer Architecture Letters, 1(1), 7-7.
[3] David J Lilja. (1993). Cache coherence in large-scale shared-memory multiprocessors: Issues and comparisons. ACM Computing Surveys (CSUR), 25(3), 303-338.
2020年度BenchCouncil成就奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020)
获奖理由“其学术成就具有里程碑意义:他不仅打造了业界首套并行基准测试套件‘Genesis’,为分布式内存并行机的性能评估奠定基础;更作为核心作者之一,参与制定了 MPI 消息传递标准的奠基性草案(与 Jack Dongarra 等合作);其近期推动的大规模数据科学基准测试工作,亦持续引领着领域发展。”
代表性论文
[1] Cliff Addison, Vladimir Getov, Tony Hey, Roger W. Hockney, I.C. Wolton. (1993). The GENESIS distributed-memory benchmarks. Elsevier.
[2] Jeyan Thiyagalingam, Mallikarjun Shankar, Geoffrey Fox, & Tony Hey. (2022). Scientific machine learning benchmarks. Nature Reviews Physics, 4(6), 413-420.
[3] Jack Dongarra, Rolf Hempei, Tony hey, David Walker. A preliminary draft proposal of MPI. November 1992.
2019年度BenchCouncil成就奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
BenchCouncil 新星奖旨在发掘并激励在基准测试及相关领域崭露头角的青年才俊。除 1000 美元奖金外,获奖者还将被授予 BenchCouncil 高级会员这一终身荣誉
获奖理由“表彰其在自然语言处理评估与基准测试领域的开创性工作,特别是创建了 SentEval、Adversarial NLI 及 Dynabench 等一系列对社区产生深远影响的基准工具。”
代表性论文
有关 Douwe Kiela博士的突出贡献详情,敬请参阅此处:
https://douwekiela.github.io/
https://scholar.google.com/citations?user=Q0piorUAAAAJ&hl=en
https://scholar.google.com/citations?user=Q0piorUAAAAJ&hl=en&oi=ao
[1] Conneau, Alexis and Douwe Kiela. “SentEval: An Evaluation Toolkit for Universal Sentence Representations.” ArXiv abs/1803.05449 (2018): n. pag.
[2] Nie, Yixin, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston and Douwe Kiela. “Adversarial NLI: A New Benchmark for Natural Language Understanding.” ACL (2020).
[3] Kiela, Douwe, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts and Adina Williams. “Dynabench: Rethinking Benchmarking in NLP.” ArXiv abs/2104.14337 (2021): n. pag.
2022年度BenchCouncil新星奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Torsten Hoefler, ETH Zürich (Since 2021)
Prof. Vijay Janapa Reddi, Harvard University (Since 2022)
Dr. Peter Mattson, Google, USA (Since 2022)
Dr. Wanling Gao, ICT, Chinese Academy of Sciences (pending)
获奖理由
Peter Mattson博士,谷歌高级工程师:
获奖理由:作为首席研究员,首创人工智能训练基准测试体系,并主导开展了大规模工业级验证;共同提出了内存访问调度技术,通过重排序内存引用,充分挖掘三维内存结构的局部性优势。
Vijay Janapa Reddi博士,哈佛大学副教授:
获奖理由:作为首席研究员,开创人工智能推理基准测试标准,并完成跨行业的规模化测试部署;共同研发了 Pin 动态插桩工具,实现了高度定制化的程序分析与性能调优。
高婉铃博士,中国科学院副研究员:
获奖理由:作为核心骨干研究员,构建了覆盖AI场景、AI训练及高性能计算(HPC)的全栈基准测试集;创新提出数据主题抽象理论,首次尝试统一大数据与人工智能的工作负载表征。
代表性论文
Peter Mattson博士,谷歌高级工程师:
[1] Rixner, S., Dally, W. J., Kapasi, U. J., Mattson, P., & Owens, J. D. (2000). Memory access scheduling. ACM SIGARCH Computer Architecture News, 28(2), 128-138.
[2] Mattson, P., Cheng, C., Diamos, G., Coleman, C., Micikevicius, P., Patterson, D., ... & Zaharia, M. (2020). Mlperf training benchmark. Proceedings of Machine Learning and Systems, 2, 336-349.
[3] Mattson, P., Reddi, V. J., Cheng, C., Coleman, C., Diamos, G., Kanter, D., ... & Wu, C. J. (2020). MLPerf: An industry standard benchmark suite for machine learning performance. IEEE Micro, 40(2), 8-16.
Vijay Janapa Reddi博士,哈佛大学副教授:
[1] Luk, C. K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., ... & Hazelwood, K. (2005). Pin: building customized program analysis tools with dynamic instrumentation. Acm sigplan notices, 40(6), 190-200.
[2] Reddi, V. J., Cheng, C., Kanter, D., Mattson, P., Schmuelling, G., Wu, C. J., ... & Zhou, Y. (2020, May). Mlperf inference benchmark. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) (pp. 446-459). IEEE.
[3] Kanev, S., Darago, J. P., Hazelwood, K., Ranganathan, P., Moseley, T., Wei, G. Y., & Brooks, D. (2015, June). Profiling a warehouse-scale computer. In Proceedings of the 42nd Annual International Symposium on Computer Architecture(pp. 158-169).
高婉铃博士,中国科学院副研究员:
[1] Gao, W., Tang, F., Zhan, J., Wen, X., Wang, L., Cao, Z., ... & Jiang, Z. (2021, September). Aibench scenario: Scenario-distilling ai benchmarking. In 2021 30th International Conference on Parallel Architectures and Compilation Techniques (PACT) (pp. 142-158). IEEE.
[2] Gao, W., Zhan, J., Wang, L., Luo, C., Zheng, D., Tang, F., ... & Ren, R. (2018, November). Data motifs: A lens towards fully understanding big data and ai workloads. In Proceedings of the 27th International Conference on Parallel Architectures and Compilation Techniques (pp. 1-14).
[3] Tang, F., Gao, W., Zhan, J., Lan, C., Wen, X., Wang, L., ... & Ye, H. (2021, March). AIBench training: Balanced industry-standard AI training benchmarking. In 2021 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) (pp. 24-35). IEEE.
2021年度BenchCouncil新星奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Torsten Hoefler, ETH Zürich (Since 2021)
获奖理由
“其学术成就涵盖三大突破:一是攻克任意拓扑下的路由难题,设计出当时最快的算法;二是作为核心作者参与制定最新的 MPI 国际标准;三是推出 Deep500 项目,打通了深度学习框架与高性能计算 AI 基准测试的壁垒。”
代表性论文
[1] Domke, J., Hoefler, T., & Nagel, W. E. (2011, May). Deadlock-free oblivious routing for arbitrary topologies. In 2011 IEEE International Parallel & Distributed Processing Symposium(pp. 616-627). IEEE.
[2] Ben-Nun, T., Besta, M., Huber, S., Ziogas, A. N., Peter, D., & Hoefler, T. (2019, May). A modular benchmarking infrastructure for high-performance and reproducible deep learning. In 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 66-77). IEEE.
[3] Gropp, W., Hoefler, T., Thakur, R., & Lusk, E. (2014). Using advanced MPI: Modern features of the message-passing interface. MIT Press.
2020年度BenchCouncil新星奖评选委员会
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
BenchCouncil 杰出博士论文奖下设计算机体系结构与其他领域两个赛道。其中,计算机体系结构赛道设有独立的提名提交通道及评审分委会。各赛道均鼓励候选人向BenchCouncil Transactions on Benchmarks, Standards, and Evaluation (TBench)投稿(详见下文投稿指南)。每个赛道将遴选出 4 名入围者,邀请其在Bench大会上作 30 分钟学术报告,并在 TBench 发表研究论文。最终,各赛道将从 4 名入围者中评定 1 名获奖者,颁发 1000 美元奖金。
需注意,两个赛道在参评资格、投稿指南、截止日期及评审标准方面规则一致。
本奖项旨在表彰并激励计算机体系结构领域内,从事基准测试、工作负载特征描述及评估研究的博士生展现卓越的研究能力与写作水平。
在线提名通道:Sumbmission Site(截止日期:2023年10月15日(全球任一时区,AoE))
奖项评审分委会
2022年度BenchCouncil计算机体系结构杰出博士论文奖
本奖项旨在表彰并激励广义基准测试、数据、标准、评估及优化领域的博士生展现卓越的研究能力与写作水平。
在线提名通道:Sumbmission Site (Deadline: October 15, 2023 End of Day, AoE)
奖项评审委员会
2021年度BenchCouncil杰出博士论文奖
委员会欢迎但不限于以下领域的提名:体系结构、系统、数据库、高性能计算、机器学习与人工智能、科学计算、医学及其他相关学科。
参评资格
学位时限:仅限在过去两年内获得博士学位者申请。
论文版本:仅接受已通过学位答辩并最终归档的博士论文终稿。
提名权限:由候选人本人或其博士生导师提名;每篇论文在整个评奖周期内仅可被提名一次。
开源要求:论文的核心贡献(包括基准测试程序、数据集或工具)必须开源。
回避制度:评审委员会及分委会成员不得提名其指导的学生。
提交流程
截止日期:2023年10月15日(全球任一时区,AoE)
在线提名通道: [计算机体系结构赛道] | [其他领域赛道]
基本信息:候选人及其导师的姓名、地址、电话及电子邮箱、候选人所属机构(填写学校全称)
推荐引文:不超过 25 个单词的简明陈述,概述候选人的关键技术或专业成就。注:最终获奖表述将由评审委员会裁定。
提名陈述:200-300 单词。重点阐述论文的重要性与独创性,而非简单重复摘要内容。
论文副本:需包含不超过 3000 单词的英文摘要。
推荐信: 至少两封来自领域内的专家推荐信,以证明论文影响力(提名人或导师不得撰写推荐信);信件需包含推荐人的姓名、地址及电话;打包要求:请将所有推荐信合并为一个 PDF 文件上传。
文章投稿指南
所有候选人必须向 BenchCouncil Transaction on Benchmarks, Standards, and Evaluations (TBench) 投稿。
稿件性质:鉴于论文内容可能已发表,允许提交综述性文章 (Survey Article)。
格式要求:必须使用 TBench 官方模板(LaTeX模板下载 )
作者署名:可仅署候选人本人,或包含导师(候选人须为第一作者)。双盲评审:投稿时需隐去所有作者信息。
内容结构(建议占比):(10%) 研究问题:论文解决的核心难题及其重要性。 (30%) 现状综述:对当前技术与实践前沿的总结。 (40%) 创新贡献:提出的创新方法、系统、工具及核心洞见。 (20%) 未来展望:开放性问题与后续工作。
投稿系统:请直接通过 TBench Editorial Manager 提交。
评审标准
评审将重点关注研究的技术深度、贡献的重要性以及对理论与实践的潜在影响。
评审流程:
初审:各赛道遴选 4 名入围者。
终审:入围者将在 Bench大会的博士论坛上进行 30 分钟现场汇报(由委员会主持)。
授奖:各赛道最终决出 1 名获奖者,颁发 1000 美元奖金。
颁奖典礼:奖项将于 Bench大会的颁奖晚宴上正式颁发。
本奖项旨在表彰在 BenchCouncil 系列会议上发表、并对基准测试、数据、标准、评估及优化领域的研究与实践具有潜在影响力的优秀论文,获奖者将获得 1000 美元奖金。自 2021 年起,本奖项仅授予第一作者为非学生身份的论文。
An Analysis of Long-tailed Network Latency Distribution and Background Traffic on Dragonfly+
Majid Salimi Beni (University of Salerno), Biagio Cosenza (University of Salerno)
Comparative Evaluation of Deep Learning Workload for Leadership-class Systems
Junqi Yin (Oak Ridge National Laboratory), Aristeidis Tsaris (Oak Ridge National Laboratory), Sajal Dash (Oak Ridge National Laboratory), Ross Miller (Oak Ridge National Laboratory), Feiyi Wang (Oak Ridge National Laboratory), Arjun Shankar (Oak Ridge National Laboratory)
Characterizing the Sharing Behavior of Applications using Software Transactional Memory
Douglas Pereira Pasqualin (Universidade Federal de Pelotas), Matthias Diener (University of Illinois at Urbana-Champaign), André Rauber Du Bois (Universidade Federal de Pelotas) and Mauricio Lima Pilla (Universidade Federal de Pelotas)
swRodinia: A Benchmark Suite for Exploiting Architecture Properties of Sunway Processor
Bangduo Chen (Beihang University), Mingzhen Li (Beihang University), Hailong Yang (Beihang University), Zhongzhi Luan (Beihang University), Lin Gan (Tsinghua University), Guangwen Yang (Tsinghua University) and Depei Qian (Beihang University)
Performance Analysis of GPU Programming Models using the Roofline Scaling Trajectories
Khaled Ibrahim, Samuel Williams and Leonid Oliker (Lawrence Berkeley National Laboratory Researchers Won BenchCouncil Bench'19 Best Paper Award)
RISC-V Track First Prize: Jiageng Yu, Yuxia Miao, Yang Tai (Institute of Software, Chinese Academy of Sciences)
RISC-V Track Second Prize: Yangyang Kong (Institute of Information Engineering, Chinese Academy of Sciences)
Cambricon Track First Prize: Guangli Li, Xueying Wang, Xiu Ma (ICT, CAS)
Cambricon Track Second Prize: Zihan Jiang, Jiansong Li (ICT, CAS)
Cambricon Track Second Prize: Yifan Wang, Chen Zeng, Chundian Li (ICT, CAS)
Cambricon Track Third Prize: Peng He, Ge Chen, Kai Deng ((ICT, CAS))
X86 Track First Prize: Weixin Deng, Jing Wang, Pengyu Wang (Shanghai Jiao Tong University)
X86 Track Second Prize: Tianshu Hao (ICT, CAS), Ziping Zheng (Google)
X86 Track Second Prize: Maosen Chen (360), Qianyun Chen (Georgia Institute of Technology), Tun Chen (ICT, CAS)
X86 Track Third Prize: Yi Liang, Shaokang Zeng, Yande Liang, Kaizhong Chen (Beijing University of Technology)
Algorithm Track First Prize: Xingwang Xiong, Xu Wen, Cheng Huang (ICT, CAS)
Algorithm Track Second Prize: Tongyan Gong (ICT, CAS), Huiqian Niu (JD.com)
Algorithm Track Second Prize: Heming Sun, Xi Xiong (The Ohio State University)
承蒙 Tony Hey 教授慷慨捐资,BenchCouncil 奖项委员会特此设立“最佳学生论文奖”。本奖项将授予以第一作者身份在 BenchCouncil 会议上发表论文,且在基准测试、度量与优化领域展现卓越潜力的学生,奖金为 1000 美元。 Tony Hey 教授现任英国卢瑟福·阿普尔顿实验室(STFC)首席数据科学家,同时是 ACM 会士、美国科学促进会(AAAS)会士及英国皇家工程院院士。他也是 2019 年度 BenchCouncil 成就奖的获得者。
MSDBench: Understanding the Performance Impact of Isolation Domains on Microservice-based IoT Deployments
Sierra Wang (University of California, Santa Barbara), Fatih Bakir (University of California, Santa Barbara), Tyler Ekaireb (University of California, Santa Barbara), Jack Pearson (University of California, Santa Barbara), Chandra Krintz (University of California, Santa Barbara), Rich Wolski(University of California, Santa Barbara)
Latency-Aware Automatic CNN Channel Pruning with GPU Runtime Analysis
Jiaqiang Liu (University of Science and Technology of China), Jingwei Sun (University of Science and Technology of China), Zhongtian Xu (University of Science and Technology of China), Guangzhong Sun (University of Science and Technology of China)
BenchCouncil 致力于孵化并托管基准测试开源项目,积极倡导基于公开基准开展可靠且可复现的科学研究。为此,特设立“可复现性研究卓越奖”,以表彰在上述项目相关研究中产出优秀成果的论文。 (奖金:每篇 100 美元,年度获奖上限:30 篇)
Characterizing the Sharing Behavior of Applications using Software Transactional Memory
Douglas Pereira Pasqualin (Universidade Federal de Pelotas), Matthias Diener (University of Illinois at Urbana-Champaign), André Rauber Du Bois (Universidade Federal de Pelotas) and Mauricio Lima Pilla (Universidade Federal de Pelotas)
swRodinia: A Benchmark Suite for Exploiting Architecture Properties of Sunway Processor
Bangduo Chen (Beihang University), Mingzhen Li (Beihang University), Hailong Yang (Beihang University), Zhongzhi Luan (Beihang University), Lin Gan (Tsinghua University), Guangwen Yang (Tsinghua University) and Depei Qian (Beihang University)
MAS3K: An Open Dataset for Marine Animal Segmentation
Lin Li (Ocean University of China), Eric Rigall (Ocean University of China), Junyu Dong (Ocean University of China) and Geng Chen (Inception Institute of Artificial Intelligence, United Arab Emirates)