Bio

The International Open Benchmark Council (BenchCouncil) is a non-profit international organization that aims to benchmark and evaluate state-of-the-art and state-of-the-practice technologies.

Since its founding, BenchCouncil bears four fundamental responsibilities: develop universal science and engineering of evaluation across different disciplines, which we call evaluatology and benchmarkology; keep the benchmarks and evaluations community open, inclusive, and growing; define emerging or future multidisciplinary and interdisciplinary challenges using benchmarks; evaluate technology achievements based on evidence.

As a non-profit organization, BenchCouncil needs your support to sustain its development. You are welcome to contribute to BenchCouncil's development by purchasing BenchCouncil's commercial tools and services.

Bio

Dr. Dhabaleswar K. (DK) Panda is a Professor and Distinguished Scholar of Computer Science at the Ohio State University. He obtained his Ph.D. in computer engineering from the University of Southern California. His research interests include high-performance computing, high-performance networking (InfiniBand), big data analytics (Spark and Hadoop), Deep Learning, cloud computing, Virtualization, GPUs and accelerators, file systems and storage, and exascale computing. He has published over 500 papers in major journals and international conferences related to these research areas. Dr. Panda has served (or serving) as Program Chair/Co-Chair/Vice Chair of many international conferences and workshops including HPCAsia '23, CCGRid '22, SCAsia '22, SCAsia '20, ISC '20, CCGrid '18, ExaComm (15-23), ESPM2 (15-23), HPBDC (15-19), CCGrid '16, PGAS '15, HPBDC '15, HiPC '12, CCGrid '12, HiPC '11, IEEE Cluster (Cluster)'10, Supercomputing (SC)'08, ANCS '07, Hot Interconnect 2007, IPDPS '07, HiPC '07, Hot Interconnect 2006, CAC (2001-04), ICPP '01, CANPC (1997-98) and ICPP '98. He has served as the General Chair/Co-Chair of CCGrid '20, IEEE Micro '19, and ICPP '06. He has served as an Associate Editor of IEEE Transactions on Parallel and Distributed Systems (TPDS), IEEE Transactions on Computers (TC), and Journal of Parallel and Distributed Computing (JPDC). Currently, he is serving as a Co-Editor-in-Chief of CCF Transactions on High-Performance Computing. He has served as Program Committee Member for more than 190 international conferences and workshops. Prof. Panda is a motivated speaker. He has Served as an IEEE Distinguished Visitor and an IEEE Chapters Tutorial Speaker. He has delivered a large number of invited Keynote/Plenary Talks, Tutorials and Presentations Worldwide.

Dr. Panda and his research group members have been doing extensive research on modern networking technologies including InfiniBand, Omni-Path, iWARP, AWS EFA, and RoCE. His research group is currently collaborating with National Laboratories and leading InfiniBand, Omni-Path, iWARP and RoCE companies on designing various subsystems of next generation high-end systems. The MVAPICH (High Performance MPI and MPI+PGAS over InfiniBand, iWARP and RoCE with support for GPGPUs and Virtualization) software libraries , developed by his research group, are currently being used by more than 3,300 organizations worldwide (in 90 countries). These software packages have enabled several InfiniBand clusters to get into the latest TOP500 ranking. As of May '23, more than 1.68M downloads of this software have taken place from the project website alone. These software packages are also available with the software stacks for network vendors (InfiniBand, Omni-Path, RoCE, AWS EFA, iWARP, and Slingshot), server vendors, packages (OpenHPC and Spack), and Linux distributors (such as RedHat and SuSE). This software is currently powering the #7 supercomputer in the world.

Multiple software libraries for Big Data processing and management, designed and developed by the group under High-Performance Big Data (HiBD) Project are available. These include: 1) MPI4Dask for data-science applications, 2) MPI4Spark for big data analytics applications; 3) RDMA-enabled Apache Hadoop Software library providing native RDMA (InfiniBand Verbs and RoCE) support for multiple components (HDFS, MapReduce and RPC) of Apache Hadoop; 4) RDMA-enabled Spark Software library providing native RDMA (InfiniBand Verbs and RoCE) support; 5) RDMA-Memcached Software library for providing native RDMA (InfiniBand Verbs and RoCE) support for Memcached used in Web 2.0 environment; and 5) OSU High-performance Big data Benchmarks (OHB). Sample performance numbers and download instructions for these packages are available from the above-mentioned website. These libraries are currently being used by more than 355 organizations worldwide (in 39 countries). As of May '23, more than 47,400 downloads of this software have taken place from the project website alone.

The group has also been focusing on accelerating Deep Learning (DL) Frameworks (TensorFlow and PyTorch) and Machine Learning (ML) Frameworks on modern HPC clusters and supercomputers. MPI-driven approaches to high-performance and scalable versions of the DL/ML frameworks are available from High-Performance Deep Learning (HiDL) Project site.

Dr. Panda leads Network-Based Computing Research Group . Students and staff members of this group are involved in multiple state-of-the-art research projects . Members of his group have obtained a large number of Awards and Recognitions . Dr. Panda's research is supported by funding from US National Science Foundation, US Department of Energy, US Department of Defense, Ohio Board of Regents, Ohio Department of DEvelopment and several industry including AMD, ARM, Broadcom, IBM, Intel, Cisco, Cornelis Networks, Cray, Oracle, SUN, Mellanox, Microsoft, NVIDIA, Pattern Computer, Rockport Networks, QLogic and NetApp.

Abstract

Performance analysis and modeling is of critical importance to computer systems and architecture research and development. We must design and build our simulators, benchmarks, and analysis tools correctly, and we must measure and analyze our performance results rigorously, otherwise experimental research and development may lead to incorrect and misleading conclusions and ineffective optimizations. These tools are critical to our understanding of both the problems and the solutions. In this talk, I will revisit the importance of rigorous performance evaluation, and decompose the performance evaluation challenge into two sub-problems, experimental design and data analysis. I will discuss some of the (not so obvious) pitfalls in both experimental design and data analysis, and argue for potential solutions. I will also emphasize the importance of picking the right level of abstraction for steering performance analysis tool as there no one size fits all. 

Bio

Lieven Eeckhout (PhD 2002) is a Senior Full Professor at Ghent University, Belgium, in the Department of Electronics and Information Systems (ELIS). His research interests include computer architecture, with specific emphasis on performance evaluation and modeling, dynamic resource management, CPU/GPU microarchitecture, and sustainability. He is the recipient of the 2017 ACM SIGARCH Maurice Wilkes Award and the 2017 OOPSLA Most Influential Paper Award, and he was elevated to IEEE Fellow in 2018 and ACM Fellow in 2021. Other awards include three IEEE Micro Top Pick selections (2007, 2010 and 2022), the ISPASS 2013 and MICRO 2023 Best Paper Awards, and Best Paper Nominations at PACT 2014, ISPASS 2012, ISPASS 2014, ISPASS 2015, ISPASS 2016, MICRO 2019, MICRO 2021 and ISCA 2023. He served as the Program Chair for ISCA 2020, HPCA 2015, CGO 2013 and ISPASS 2009, and has served or serves as General Chair for ISPASS 2010, IISWC 2023 and ASPLOS 2025. He served as the Chair of the IEEE Computer So

简介

摘要

简介

摘要

Sanya, also known as "China's Hawaii," is a breathtaking destination renowned for its stunning natural scenery, pleasant climate, and pristine air quality. Located on the southern coast of Hainan Island, Sanya boasts a perfect combination of picturesque landscapes and a tropical paradise. The climate in Sanya is pleasant, with a perpetual spring-like atmosphere, and one can escape the cold winter elsewhere and indulge in the warm sunshine, pristine beaches, crystal-clear waters, delectable cuisine, and lush tropical landscapes.

The Travel Grant is a compensatory award that is available to applicants who have registered for the conference. The conference will be held Dec.3 – Dec.6 in Sanya, Hainan, China, all registers are eligible to apply. Since the total grant is a fixed amount, the amount that can be awarded to each register will depend on the number of awardees and will cover only a portion of your expenses.

There are two levels of Travel Grant: 2100 RMB (300 USD) for up to 20 people, and 700 RMB (100 USD) for up to 100 people. The final list of recipients of the Travel Grant will be announced during the conference banquet. Please note that individuals who have already applied for the family meal ticket are not eligible to apply for the Travel Grant.

Important information about travel grants:

1. Recipients do not need to be authors/presenters to qualify for a travel grant.
2. Travel Grant Application Deadline: Nov 30, 2023.
3. Grantees must save original receipts and mail them in or present them at the conference to verify their expenditures.
4. Expenses will be reimbursed after the conference.
5. Awardees will be required to email the travel grant chair a short essay describing their Bench experience.

If you wish to apply for the grant, please contact student.travel.grant.bench@gmail.com.

Use the subject line "Bench 2023 Travel Grant Application." The travel grant application (a single consolidated pdf file) should include:

1. A write-up stating the reason for attending Bench and a description of research interests. This should include information about whether you are presenting work at the conference.
2. A resume of no more than One pages

BenchCouncil has launched the Top100 initiative, which aims to recognize the top 100 achievements in the fields of chips, AI, open source, evaluations, and benchmarks. These achievements are known as Chip100, AI100, Open100, and Bench100. As a result, we have compiled both annual and centennial versions of four rankings, which can be accessed publicly at https://www.benchcouncil.org/evaluation/. These rankings have generated significant interest, comments, and suggestions from academic and industry practitioners worldwide.

For those that have been selected into annual version of the BenchCouncil Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100)

• • The selection process was highly competitive, with only 100 achievements chosen from tens of thousands of candidates. As a token of appreciation, BenchCouncil will provide you with a complimentary registration and a standard-size exhibition space for two days in the FICC conference. The FICC conference consists of four concurrent symposiums: Chips 2023, Bench 2023, OpenCS 2023, and IC 2023.

---

Complimentary service

---

Academia complimentary Service:

• • 1 complimentary registration for FICC 2023.
• • 20-minute speech at the regular session in one of symposium (Chips or Bench or OpenCS or IC).
• • Standard-size exhibition space for 2 days.

Industry complimentary Service:

• • 1 complimentary registration for FICC 2023.
• • 20-minute regular session.
• • Standard-size exhibition space for 2 days.

Free registration: https://eur.cvent.me/eAxmP

---

Exhibition

---

For every selected achievement, BenchCouncil will provide you with a standard-size exhibition space (2.5m*1m) if you need it. Please note that we try to provide tables and chairs if possible. However, we cannot guarantee that. If you need roll-up banners, please prepare them yourself, and the size should be 2m*0.8m. [Roll-up Banner Template]

---

The other benefits

---

• • The certificate of the 2022-2023 Top 100 Achievement Award.
• • BenchCouncil will select ten achievements from the ranking and present the 2023 Top 10 Achievement Award through the evaluation committee's selection process and the candidates from the annual ranking list.
• • the FICC 2023 conference will attract a wide range of investors and venture capitalists, providing an opportunity for technologists to showcase their projects and explore potential investment opportunities to support their business growth.
• • BenchCouncil has plans to expand the centennial edition of the Top 100 achievement ranking to a centennial edition of the Top 500 ranking in the near future, and you are, of course, one of the strong candidates.

---

Furthermore, we offer upgrade services for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. We can also provide customized services upon request.

If you need to register for upgrade service, please transfer the cash to the following bank account.

• • Account Name: HONG KONG AI AND CHIP BENCHMARK RESEARCH LIMITED
• • Account Number: 801-670811-838
• • Account Address: 6/F Manulife Place, 348 Kwun Tong Road, Kowloon, Hong Kong.
• • Bank Name: The Hongkong and Shanghai Banking Corporation Limited
• • Bank Address: HSBC Main Building, 1 Queen's Road Central, Central, Victoria City, Hong Kong.
• • SWIFT Code: HSBCHKHHHKH

Meanwhile, please send an email to benchcouncil.evaluation@gmail.com. Also, you can upgrade services through the registration link https://eur.cvent.me/eAxmP.

---

the default upgrade services are as follows

---

Academia Upgrade Service:

• • Academia Bronze Service:
  1. USD $1000.
  2. 2 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 2 days.


• • Academia Silver Service:
  1. USD $3000.
  2. 4 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.


• • Academia Golden Service:
  1. USD $4000.
  2. 4 free registrations for FICC 2023.
  3. 30-minute primary session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.


• • Academia Platinum Service:
  1. USD $5000.
  2. 6 free registrations for FICC 2023.
  3. 30-minute plenary session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.

---

Industry Upgrade Service:

• • Industry Bronze Service:
  1. USD $2000.
  2. 2 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 2 days.


• • Industry Silver Service:
  1. USD $4000.
  2. 4 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 4 days.


• • Industry Golden Service:
  1. USD $8000.
  2. 6 free registrations for FICC 2023.
  3. 30-minute primary session.
  4. Double standard-size exhibition space for 4 days.


• • Industry Platinum Service:
  1. USD $10000.
  2. 8 free registrations for FICC 2023.
  3. 30-minute plenary session.
  4. Four standard-size exhibition spaces for 4 days.

To avoid missing important achievements, BenchCouncil calls for the submission of achievements in the fields of AI, Chips, Open source, benchmarks, and evaluations. After the preliminary evaluation, the submitters of the achievements will have the opportunity to present or showcase their work at the FICC 2023 conference and have a chance to be included in the final rankings.

Ten benefits

1. The submitted achievements will be included in the annual achievement report of the BenchCouncil and will be cited. The submitters need to provide valid citation formats, including paper publications, code repositories, etc. The compiled annual achievement report will be officially published in BenchCouncil Transactions.

2. The submitters of the achievements will receive the certificate of annual achievement Selection. For example, the certificate for the Chip category will be named "the certificate of 2023 BenchCouncil Chip achievement selection." The certificate will display the individual and institutional names.

3. The achievements will be featured on the official English webpages of the BenchCouncil's achievement evaluation (https://www.benchcouncil.org/evaluation/).

4. The official English webpages of the BenchCouncil will provide links to the achievements. The submitters should ensure that the linked content is authentic, non-deceptive, and accurate.

5. The submitters will have the opportunity to attend all four conferences: Chips 2023, OpenCS 2023, Bench 2023, and IC 2023. They will have the chance to engage in extensive exchanges with international peers.

6. The selected achievements will be showcased at the conference venue. The specific time period (generally not exceeding one day) and location need to be coordinated with the conference organizers.

• • For every selected achievement, BenchCouncil will provide you with a standard-size exhibition space (2.5m*1m) if you need it. Please note that we try to provide tables and chairs if possible. However, we cannot guarantee that. If you need roll-up banners, please prepare them yourself, and the size should be 2m*0.8m. [Roll-up Banner Template]

7. The submitters of the achievements have the opportunity to present their work at the conference, subject to further evaluation and arrangement by the conference organizers.

8. Outstanding achievements invited to present at the conference may have the chance to be included in the annual version of the Top 100 achievements ranking (final version) after evaluation by the committee and recognition from peers. The committee has reserved 20 slots for selecting achievements from the submissions.

9. Exemplary achievements that have stood the test of time have the chance to be included in the annual version of the Top 100 achievements ranking (request for comments).

10. The submitters of the achievements can apply for travel funding.

---

The summary of BenchCouncil awards

---

Scope of achievements:

In the field of chips, including but not limited to as follows:

system design, logic design, physical design, timing design, verification and simulation, emulators, auxiliary design tools, emerging chips such as superconducting and quantum chips, emerging accelerators, semiconductor materials, lithography technology, circuits, packaging technology.

In the field of artificial intelligence, including but not limited to as follows:

machine learning, reinforcement learning, natural language processing, image analysis, video analysis, data mining, recommendation systems, knowledge representation and reasoning, medical image processing, multimodal information fusion, collective intelligence, intelligent robots and systems, intelligent control, intelligent healthcare, artificial intelligence and astronomy, artificial intelligence and high-energy physics, artificial intelligence and space science, artificial intelligence and transportation, artificial intelligence and oceanography, artificial intelligence and security, artificial intelligence and law, artificial intelligence and finance, artificial intelligence and traditional industries, ethics and governance of artificial intelligence, artificial intelligence and big data, artificial intelligence and materials, artificial intelligence and civil aviation applications, artificial intelligence and business management.

In the field of open source, including but not limited to as follows:

chips, operating systems, containerization and virtualization, compilers, data management, networks, programming languages and compilers, systems and frameworks, basic libraries, monitoring and optimization tools, big data, AI, cloud computing, HPC, blockchain, LLM, autonomous driving, digital twins, Internet of Things, privacy and security, datasets, education.

In the field of benchmark and evaluation, including but not limited to as follows:

econometrics, clinical medical evaluation, drug evaluation, business and financial benchmarks, AI evaluation, HPC benchmarks, database benchmarks, memory benchmarks, network benchmarks, hardware and disk benchmarks, graph benchmarks, human resource evaluation, education evaluation, indicator and index systems.

Process of submitting achievements:

• 1. Please submit the achievements to the email address benchcouncil.evaluation@gmail.com. The materials should not exceed one page and should include a description of the achievements and supporting materials (including achievement links). Please also provide the names of the contributors and contact information.
• 2. After the preliminary evaluation, you will be notified by email within one working day. Please keep the materials concise.
• 3. If you have published papers at Bench 2023 and IC 2023, your achievements will be automatically included after submitting the supporting materials.
• 4. The submitters of the achievements need to sign the conference participation and safety briefing. At least one person is required to register for the conference and showcase the achievements (poster) during a specific time period. Student submitters can register for the conference as students. There are no additional fees for the conference, except for upgrade services.

Evaluation criteria:

• 1. The achievements can be in the form of published or preprint papers, open-source code, products, or services.
• 2. You must submit an English document describing your achievements.
• 3. In order to encourage more participation, the acceptance rate for the achievements should not be lower than 80%.

---

Registration

---

After being selected, you must register the conference.

please transfer the cash to the following bank account.

• • Account Name: HONG KONG AI AND CHIP BENCHMARK RESEARCH LIMITED
• • Account Number: 801-670811-838
• • Account Address: 6/F Manulife Place, 348 Kwun Tong Road, Kowloon, Hong Kong.
• • Bank Name: The Hongkong and Shanghai Banking Corporation Limited
• • Bank Address: HSBC Main Building, 1 Queen's Road Central, Central, Victoria City, Hong Kong.
• • SWIFT Code: HSBCHKHHHKH

Also, You can pay using your credit card with following registration link https://eur.cvent.me/5qQEq

1. 1 standard registration for FICC 2023.

• • USD $490.00 by December 3rd. USD $550.00 after December 3rd.
• • Or 1 simple registration not including paper publication (No food and beverage included)
• • US $320.00 by December 3rd. USD $360 after December 3rd.

2. Standard-size exhibition space for 1 day.

Furthermore, we offer upgrade services (the registration link is: https://eur.cvent.me/k85BK) for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. Also, we provide customized services upon request.

---

the default upgrade services are as follows

---

Academia Upgrade Service
Service Level	Bronze Service	Silver Service	Golden Service	Platinum Service
Expense	USD $1000	USD $3000	USD $4000	USD $5000
Free Registration	2 free registrations for FICC 2023	4 free registrations for FICC 2023	4 free registrations for FICC 2023	6 free registrations for FICC 2023
Session Time	20-minute regular session	20-minute regular session	30-minute primary session	30-minute plenary session
Exhibition	Double standard-size exhibition space for 2 days	Double standard-size exhibition space for 4 days	Double standard-size exhibition space for 4 days	Double standard-size exhibition space for 4 days
Evaluation Report		5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information	5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information	5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information

---

Industry Upgrade Service
Service Level	Bronze Service	Silver Service	Golden Service	Platinum Service
Expense	USD $2000	USD $4000	USD $8000	USD $10000
Free Registration	2 free registrations for FICC 2023	4 free registrations for FICC 2023	6 free registrations for FICC 2023	8 free registrations for FICC 2023
Session Time	20-minute regular session	20-minute regular session	30-minute primary session	30-minute plenary session
Exhibition	Double standard-size exhibition space for 2 days	Double standard-size exhibition space for 4 days	Double standard-size exhibition space for 4 days	Four standard-size exhibition spaces for 4 days