Bio

The International Open Benchmark Council (BenchCouncil) is a non-profit international organization dedicated to benchmarks and evaluations.

Since its establishment, BenchCouncil has embraced four core responsibilities: developing the universal sciences and engineering of evaluation across various fields, known as evaluatology and benchmarkology; fostering an open, inclusive, and growing community for benchmarks and evaluations; defining emerging or future multidisciplinary and interdisciplinary challenges through the use of benchmarks; and conducting evaluations of scientific and technological achievements based on evidence.

As a non-profit organization, BenchCouncil relies on your support to sustain its development. You are encouraged to contribute to BenchCouncil's growth by purchasing its commercial tools and services.

Bio

Dr. Dhabaleswar K. (DK) Panda is a Professor and Distinguished Scholar of Computer Science at the Ohio State University. He obtained his Ph.D. in computer engineering from the University of Southern California. His research interests include high-performance computing, high-performance networking (InfiniBand), big data analytics (Spark and Hadoop), Deep Learning, cloud computing, Virtualization, GPUs and accelerators, file systems and storage, and exascale computing. He has published over 500 papers in major journals and international conferences related to these research areas. Dr. Panda has served (or serving) as Program Chair/Co-Chair/Vice Chair of many international conferences and workshops including HPCAsia '23, CCGRid '22, SCAsia '22, SCAsia '20, ISC '20, CCGrid '18, ExaComm (15-23), ESPM2 (15-23), HPBDC (15-19), CCGrid '16, PGAS '15, HPBDC '15, HiPC '12, CCGrid '12, HiPC '11, IEEE Cluster (Cluster)'10, Supercomputing (SC)'08, ANCS '07, Hot Interconnect 2007, IPDPS '07, HiPC '07, Hot Interconnect 2006, CAC (2001-04), ICPP '01, CANPC (1997-98) and ICPP '98. He has served as the General Chair/Co-Chair of CCGrid '20, IEEE Micro '19, and ICPP '06. He has served as an Associate Editor of IEEE Transactions on Parallel and Distributed Systems (TPDS), IEEE Transactions on Computers (TC), and Journal of Parallel and Distributed Computing (JPDC). Currently, he is serving as a Co-Editor-in-Chief of CCF Transactions on High-Performance Computing. He has served as Program Committee Member for more than 190 international conferences and workshops. Prof. Panda is a motivated speaker. He has Served as an IEEE Distinguished Visitor and an IEEE Chapters Tutorial Speaker. He has delivered a large number of invited Keynote/Plenary Talks, Tutorials and Presentations Worldwide.

Dr. Panda and his research group members have been doing extensive research on modern networking technologies including InfiniBand, Omni-Path, iWARP, AWS EFA, and RoCE. His research group is currently collaborating with National Laboratories and leading InfiniBand, Omni-Path, iWARP and RoCE companies on designing various subsystems of next generation high-end systems. The MVAPICH (High Performance MPI and MPI+PGAS over InfiniBand, iWARP and RoCE with support for GPGPUs and Virtualization) software libraries , developed by his research group, are currently being used by more than 3,300 organizations worldwide (in 90 countries). These software packages have enabled several InfiniBand clusters to get into the latest TOP500 ranking. As of May '23, more than 1.68M downloads of this software have taken place from the project website alone. These software packages are also available with the software stacks for network vendors (InfiniBand, Omni-Path, RoCE, AWS EFA, iWARP, and Slingshot), server vendors, packages (OpenHPC and Spack), and Linux distributors (such as RedHat and SuSE). This software is currently powering the #7 supercomputer in the world.

Multiple software libraries for Big Data processing and management, designed and developed by the group under High-Performance Big Data (HiBD) Project are available. These include: 1) MPI4Dask for data-science applications, 2) MPI4Spark for big data analytics applications; 3) RDMA-enabled Apache Hadoop Software library providing native RDMA (InfiniBand Verbs and RoCE) support for multiple components (HDFS, MapReduce and RPC) of Apache Hadoop; 4) RDMA-enabled Spark Software library providing native RDMA (InfiniBand Verbs and RoCE) support; 5) RDMA-Memcached Software library for providing native RDMA (InfiniBand Verbs and RoCE) support for Memcached used in Web 2.0 environment; and 5) OSU High-performance Big data Benchmarks (OHB). Sample performance numbers and download instructions for these packages are available from the above-mentioned website. These libraries are currently being used by more than 355 organizations worldwide (in 39 countries). As of May '23, more than 47,400 downloads of this software have taken place from the project website alone.

The group has also been focusing on accelerating Deep Learning (DL) Frameworks (TensorFlow and PyTorch) and Machine Learning (ML) Frameworks on modern HPC clusters and supercomputers. MPI-driven approaches to high-performance and scalable versions of the DL/ML frameworks are available from High-Performance Deep Learning (HiDL) Project site.

Dr. Panda leads Network-Based Computing Research Group . Students and staff members of this group are involved in multiple state-of-the-art research projects . Members of his group have obtained a large number of Awards and Recognitions . Dr. Panda's research is supported by funding from US National Science Foundation, US Department of Energy, US Department of Defense, Ohio Board of Regents, Ohio Department of DEvelopment and several industry including AMD, ARM, Broadcom, IBM, Intel, Cisco, Cornelis Networks, Cray, Oracle, SUN, Mellanox, Microsoft, NVIDIA, Pattern Computer, Rockport Networks, QLogic and NetApp.

Abstract

Performance analysis and modeling is of critical importance to computer systems and architecture research and development. We must design and build our simulators, benchmarks, and analysis tools correctly, and we must measure and analyze our performance results rigorously, otherwise experimental research and development may lead to incorrect and misleading conclusions and ineffective optimizations. These tools are critical to our understanding of both the problems and the solutions. In this talk, I will revisit the importance of rigorous performance evaluation, and decompose the performance evaluation challenge into two sub-problems, experimental design and data analysis. I will discuss some of the (not so obvious) pitfalls in both experimental design and data analysis, and argue for potential solutions. I will also emphasize the importance of picking the right level of abstraction for steering performance analysis tool as there no one size fits all. 

Bio

Lieven Eeckhout (PhD 2002) is a Senior Full Professor at Ghent University, Belgium, in the Department of Electronics and Information Systems (ELIS). His research interests include computer architecture, with specific emphasis on performance evaluation and modeling, dynamic resource management, CPU/GPU microarchitecture, and sustainability. He is the recipient of the 2017 ACM SIGARCH Maurice Wilkes Award and the 2017 OOPSLA Most Influential Paper Award, and he was elevated to IEEE Fellow in 2018 and ACM Fellow in 2021. Other awards include three IEEE Micro Top Pick selections (2007, 2010 and 2022), the ISPASS 2013 and MICRO 2023 Best Paper Awards, and Best Paper Nominations at PACT 2014, ISPASS 2012, ISPASS 2014, ISPASS 2015, ISPASS 2016, MICRO 2019, MICRO 2021 and ISCA 2023. He served as the Program Chair for ISCA 2020, HPCA 2015, CGO 2013 and ISPASS 2009, and has served or serves as General Chair for ISPASS 2010, IISWC 2023 and ASPLOS 2025. He served as the Chair of the IEEE Computer So

简介

摘要

简介

摘要

Sanya, also known as "China's Hawaii," is a breathtaking destination renowned for its stunning natural scenery, pleasant climate, and pristine air quality. Located on the southern coast of Hainan Island, Sanya boasts a perfect combination of picturesque landscapes and a tropical paradise. The climate in Sanya is pleasant, with a perpetual spring-like atmosphere, and one can escape the cold winter elsewhere and indulge in the warm sunshine, pristine beaches, crystal-clear waters, delectable cuisine, and lush tropical landscapes.

Why should you not miss the opportunity to exchange ideas with global top tech entrepreneurs? Here are ten reasons to participate in the conference:

The 2023 BenchCouncil International Conference on Intelligent Computing and Chip (FICC 2023) will be held in Sanya from December 3rd to 6th. As an important event in this field, FICC 2023 provides a unique opportunity for companies to showcase their technologies, share their experiences, and engage in in-depth discussions with top global experts and entrepreneurs.

Here are ten reasons for you to attend FICC 2023 and deliver a presentation at the conference:

1. Global Influence: FICC 2023 is an international event that will attract top technical experts and business representatives from around the world. As a speaker, you will have the opportunity to connect with leading global companies and expand your influence.
2. Free Speeches and Exhibition Area: Unlike other conferences, FICC 2023 offers the opportunity for free speeches and exhibition booths. This means you can deliver a speech and showcase your work for free during the conference, gaining insights into the latest industry trends and solutions. Additionally, the conference offers paid upgrade services to maximize your benefits.
3. Diversified Business Expansion Channels: FICC 2023 consists of four major conferences: Chip, Artificial Intelligence, Open Source, and Evaluation & Benchmark. Participants include government officials, investment institutions, research organizations, and various types of enterprises, which will help expand your business channels. FICC 2023 will attract representatives from global companies, providing you with opportunities for business negotiations with potential partners and clients. It is an excellent platform for expanding your business network and seeking new opportunities.
4. Opportunity for Four Major Rankings: As a speaker at FICC 2023, you will have the opportunity to be selected for the four influential rankings: Chip100, Open 100, AI100, and Bench100. This will further enhance the reputation and status of you and your company within the industry. Please note that all rankings are independently reviewed.
5. Industry Leadership: FICC 2023 brings together leading experts and companies in the fields of artificial intelligence, chips, open source, and evaluation & benchmarking, providing you with a platform to understand the latest industry trends and cutting-edge technologies. Your speech will establish your company as one of the industry leaders.
6. Knowledge Sharing: FICC 2023 emphasizes the dissemination and sharing of knowledge. Through your speech, you will have the opportunity to introduce your innovative technologies, research achievements, and practical experiences to the participants, sharing and inspiring each other.
7. Brand Promotion: As a speaker, your company will gain extensive exposure. FICC 2023 will attract attention from the media, academia, and industry, effectively promoting your brand and enhancing its visibility and reputation.
8. Technical Exchange: During your speech at FICC 2023, you will have in-depth discussions with other technical experts, sharing the latest research findings and technological challenges. This will promote innovation and collaboration, contributing to the development of the industry.
9. Speaking alongside Global Outstanding Technical Professionals: FICC 2023 will invite outstanding technical professionals from around the world to give speeches. As one of them, you will have the opportunity to exchange ideas and share your insights and experiences with these top experts.
10. Beautiful Scenery in Sanya: FICC 2023 will be held in Sanya, a beautiful coastal city known for its pleasant climate and magnificent scenery. In addition to the conference, you can enjoy the scenery of Sanya and experience its unique culture and cuisine.

Participating in FICC 2023 will bring tremendous opportunities and benefits to your company. I sincerely invite you to attend this grand event and deliver a keynote speech. Please confirm your intention to attend as soon as possible so that we can provide you with more detailed information.

The application process to participate in the Sanya International Conference on Intelligent Computing and Chip Union is as follows:

1. Send the following information via email to benchcouncil.evaluation@gmail.com:
   • A 3-page Chinese or English introduction, including:
     • Core directions and product advantages of your company (1 page)
     • Brief introductions and contact information of key personnel and speakers from your company (1 page)
     • Title, abstract, and exhibition plan for the presentation (1 page)
2. The evaluation process will be completed within 2 working days.
3. If your application is approved, the conference will contact you to complete the registration process.

Please provide clear and detailed information in the email to help the evaluators fully understand your company and the content of your presentation. We wish you success in your application!

The Travel Grant is a compensatory award that is available to applicants who have registered for the conference. The conference will be held Dec.3 – Dec.6 in Sanya, Hainan, China, all registers are eligible to apply. Since the total grant is a fixed amount, the amount that can be awarded to each register will depend on the number of awardees and will cover only a portion of your expenses.

FICC 2023 Travel Grant Recipients:

1. Singh Rohit, University of Cincinnati (300 USD)
2. Xinlin Wang, University of Luxembourg (100 USD)
3. Haojia Huang, Sun Yat-sen University (100 USD)
4. Jiayi Xu, Institute of High Energy Physics, Chinese Academy of Sciences (100 USD)
5. Jiaxing Li, Nankai University (100 USD)
6. Jiahui Shen, Civil Aviation Flight University of China (100 USD)
7. Hao Wang, Tongji University (100 USD)
8. Song Li, University of Copenhagen (100 USD)
9. Cen Mo, Shanghai Jiaotong University (100 USD)
10. Zejia Lu, Shanghai Jiaotong University (100 USD)

There are two levels of Travel Grant: 2100 RMB (300 USD) for up to 20 people, and 700 RMB (100 USD) for up to 100 people. The final list of recipients of the Travel Grant will be announced during the conference banquet. Please note that individuals who have already applied for the family meal ticket are not eligible to apply for the Travel Grant.

Important information about travel grants:

1. Recipients do not need to be authors/presenters to qualify for a travel grant.
2. Travel Grant Application Deadline: Nov 30, 2023.
3. Grantees must save original receipts and mail them in or present them at the conference to verify their expenditures.
4. Expenses will be reimbursed after the conference.
5. Awardees will be required to email the travel grant chair a short essay describing their FICC experience.

If you wish to apply for the grant, please contact student.travel.grant.bench@gmail.com.

Use the subject line "FICC 2023 Travel Grant Application." The travel grant application (a single consolidated pdf file) should include:

1. A write-up stating the reason for attending FICC and a description of research interests. This should include information about whether you are presenting work at the conference.
2. A resume of no more than One pages

BenchCouncil has launched the Top100 initiative, which aims to recognize the top 100 achievements in the fields of chips, AI, open source, evaluations, and benchmarks. These achievements are known as Chip100, AI100, Open100, and Bench100. As a result, we have compiled both annual and centennial versions of four rankings, which can be accessed publicly at https://www.benchcouncil.org/evaluation/. These rankings have generated significant interest, comments, and suggestions from academic and industry practitioners worldwide.

For those that have been selected into annual version of the BenchCouncil Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100)

• • The selection process was highly competitive, with only 100 achievements chosen from tens of thousands of candidates. As a token of appreciation, BenchCouncil will provide you with a complimentary registration and a standard-size exhibition space for two days in the FICC conference. The FICC conference consists of four concurrent symposiums: Chips 2023, Bench 2023, OpenCS 2023, and IC 2023.

Complimentary service

Academia complimentary Service:

• • 1 complimentary registration for FICC 2023.
• • 20-minute speech at the regular session in one of symposium (Chips or Bench or OpenCS or IC).
• • Standard-size exhibition space for 2 days.

Industry complimentary Service:

• • 1 complimentary registration for FICC 2023.
• • 20-minute regular session.
• • Standard-size exhibition space for 2 days.

Free registration: https://eur.cvent.me/eAxmP

Exhibition

For every selected achievement, BenchCouncil will provide you with a standard-size exhibition space (2.5m*1m) if you need it. Please note that we try to provide tables and chairs if possible. However, we cannot guarantee that. If you need roll-up banners, please prepare them yourself, and the size should be 2m*0.8m. [Roll-up Banner Template]

The other benefits

• • The certificate of the 2022-2023 Top 100 Achievement Award.
• • BenchCouncil will select ten achievements from the ranking and present the 2023 Top 10 Achievement Award through the evaluation committee's selection process and the candidates from the annual ranking list.
• • the FICC 2023 conference will attract a wide range of investors and venture capitalists, providing an opportunity for technologists to showcase their projects and explore potential investment opportunities to support their business growth.
• • BenchCouncil has plans to expand the centennial edition of the Top 100 achievement ranking to a centennial edition of the Top 500 ranking in the near future, and you are, of course, one of the strong candidates.

Furthermore, we offer upgrade services for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. We can also provide customized services upon request.

If you need to register for upgrade service, please transfer the cash to the following bank account.

• • Account Name: HONG KONG AI AND CHIP BENCHMARK RESEARCH LIMITED
• • Account Number: 801-670811-838
• • Account Address: 6/F Manulife Place, 348 Kwun Tong Road, Kowloon, Hong Kong.
• • Bank Name: The Hongkong and Shanghai Banking Corporation Limited
• • Bank Address: HSBC Main Building, 1 Queen's Road Central, Central, Victoria City, Hong Kong.
• • SWIFT Code: HSBCHKHHHKH

Meanwhile, please send an email to benchcouncil.evaluation@gmail.com. Also, you can upgrade services through the registration link https://eur.cvent.me/eAxmP.

the default upgrade services are as follows

Academia Upgrade Service:

• • Academia Bronze Service:
  1. USD $1000.
  2. 2 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 2 days.


• • Academia Silver Service:
  1. USD $3000.
  2. 4 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.


• • Academia Golden Service:
  1. USD $4000.
  2. 4 free registrations for FICC 2023.
  3. 30-minute primary session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.


• • Academia Platinum Service:
  1. USD $5000.
  2. 6 free registrations for FICC 2023.
  3. 30-minute plenary session.
  4. Double standard-size exhibition space for 4 days.
  5. 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information.

Industry Upgrade Service:

• • Industry Bronze Service:
  1. USD $2000.
  2. 2 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 2 days.


• • Industry Silver Service:
  1. USD $4000.
  2. 4 free registrations for FICC 2023.
  3. 20-minute regular session.
  4. Double standard-size exhibition space for 4 days.


• • Industry Golden Service:
  1. USD $8000.
  2. 6 free registrations for FICC 2023.
  3. 30-minute primary session.
  4. Double standard-size exhibition space for 4 days.


• • Industry Platinum Service:
  1. USD $10000.
  2. 8 free registrations for FICC 2023.
  3. 30-minute plenary session.
  4. Four standard-size exhibition spaces for 4 days.

To avoid missing important achievements, BenchCouncil calls for the submission of achievements in the fields of AI, Chips, Open source, benchmarks, and evaluations. After the preliminary evaluation, the submitters of the achievements will have the opportunity to present or showcase their work at the FICC 2023 conference and have a chance to be included in the final rankings.

Ten benefits

1. The submitted achievements will be included in the annual achievement report of the BenchCouncil and will be cited. The submitters need to provide valid citation formats, including paper publications, code repositories, etc. The compiled annual achievement report will be officially published in BenchCouncil Transactions.

2. The submitters of the achievements will receive the certificate of annual achievement Selection. For example, the certificate for the Chip category will be named "the certificate of 2023 BenchCouncil Chip achievement selection." The certificate will display the individual and institutional names.

3. The achievements will be featured on the official English webpages of the BenchCouncil's achievement evaluation (https://www.benchcouncil.org/evaluation/).

4. The official English webpages of the BenchCouncil will provide links to the achievements. The submitters should ensure that the linked content is authentic, non-deceptive, and accurate.

5. The submitters will have the opportunity to attend all four conferences: Chips 2023, OpenCS 2023, Bench 2023, and IC 2023. They will have the chance to engage in extensive exchanges with international peers.

6. The selected achievements will be showcased at the conference venue. The specific time period (generally not exceeding one day) and location need to be coordinated with the conference organizers.

• • For every selected achievement, BenchCouncil will provide you with a standard-size exhibition space (2.5m*1m) if you need it. Please note that we try to provide tables and chairs if possible. However, we cannot guarantee that. If you need roll-up banners, please prepare them yourself, and the size should be 2m*0.8m. [Roll-up Banner Template]

7. The submitters of the achievements have the opportunity to present their work at the conference, subject to further evaluation and arrangement by the conference organizers.

8. Outstanding achievements invited to present at the conference may have the chance to be included in the annual version of the Top 100 achievements ranking (final version) after evaluation by the committee and recognition from peers. The committee has reserved 20 slots for selecting achievements from the submissions.

9. Exemplary achievements that have stood the test of time have the chance to be included in the annual version of the Top 100 achievements ranking (request for comments).

10. The submitters of the achievements can apply for travel funding.

The summary of BenchCouncil awards

Scope of achievements:

In the field of chips, including but not limited to as follows:

system design, logic design, physical design, timing design, verification and simulation, emulators, auxiliary design tools, emerging chips such as superconducting and quantum chips, emerging accelerators, semiconductor materials, lithography technology, circuits, packaging technology.

In the field of artificial intelligence, including but not limited to as follows:

machine learning, reinforcement learning, natural language processing, image analysis, video analysis, data mining, recommendation systems, knowledge representation and reasoning, medical image processing, multimodal information fusion, collective intelligence, intelligent robots and systems, intelligent control, intelligent healthcare, artificial intelligence and astronomy, artificial intelligence and high-energy physics, artificial intelligence and space science, artificial intelligence and transportation, artificial intelligence and oceanography, artificial intelligence and security, artificial intelligence and law, artificial intelligence and finance, artificial intelligence and traditional industries, ethics and governance of artificial intelligence, artificial intelligence and big data, artificial intelligence and materials, artificial intelligence and civil aviation applications, artificial intelligence and business management.

In the field of open source, including but not limited to as follows:

chips, operating systems, containerization and virtualization, compilers, data management, networks, programming languages and compilers, systems and frameworks, basic libraries, monitoring and optimization tools, big data, AI, cloud computing, HPC, blockchain, LLM, autonomous driving, digital twins, Internet of Things, privacy and security, datasets, education.

In the field of benchmark and evaluation, including but not limited to as follows:

econometrics, clinical medical evaluation, drug evaluation, business and financial benchmarks, AI evaluation, HPC benchmarks, database benchmarks, memory benchmarks, network benchmarks, hardware and disk benchmarks, graph benchmarks, human resource evaluation, education evaluation, indicator and index systems.

Process of submitting achievements:

• 1. Please submit the achievements to the email address benchcouncil.evaluation@gmail.com. The materials should not exceed one page and should include a description of the achievements and supporting materials (including achievement links). Please also provide the names of the contributors and contact information.
• 2. After the preliminary evaluation, you will be notified by email within one working day. Please keep the materials concise.
• 3. If you have published papers at Bench 2023 and IC 2023, your achievements will be automatically included after submitting the supporting materials.
• 4. The submitters of the achievements need to sign the conference participation and safety briefing. At least one person is required to register for the conference and showcase the achievements (poster) during a specific time period. Student submitters can register for the conference as students. There are no additional fees for the conference, except for upgrade services.

Evaluation criteria:

• 1. The achievements can be in the form of published or preprint papers, open-source code, products, or services.
• 2. You must submit an English document describing your achievements.
• 3. In order to encourage more participation, the acceptance rate for the achievements should not be lower than 80%.

Registration

After being selected, you must register the conference.

please transfer the cash to the following bank account.

• • Account Name: HONG KONG AI AND CHIP BENCHMARK RESEARCH LIMITED
• • Account Number: 801-670811-838
• • Account Address: 6/F Manulife Place, 348 Kwun Tong Road, Kowloon, Hong Kong.
• • Bank Name: The Hongkong and Shanghai Banking Corporation Limited
• • Bank Address: HSBC Main Building, 1 Queen's Road Central, Central, Victoria City, Hong Kong.
• • SWIFT Code: HSBCHKHHHKH

Also, You can pay using your credit card with following registration link https://eur.cvent.me/5qQEq

1. 1 standard registration for FICC 2023.

• • USD $490.00 by December 3rd. USD $550.00 after December 3rd.
• • Or 1 simple registration not including paper publication (No food and beverage included)
• • US $320.00 by December 3rd. USD $360 after December 3rd.

Furthermore, we offer upgrade services (the registration link is: https://eur.cvent.me/k85BK) for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. Also, we provide customized services upon request.

the default upgrade services are as follows