Bench 2023 Call For Special Sessions And Tutorial Proposals

2023 BenchCouncil International Conference on Benchmarking, Measuring and Optimizing

The Bench conference encompasses a wide range of topics in benchmarking, measurement, evaluation methods and tools. Bench’s multi-disciplinary emphasis provides an ideal environment for developers and researchers from the architecture, system, algorithm, computing, and application communities to discuss practical and theoretical work covering workload characterization, benchmarks and tools, evaluation, measurement and optimization, and dataset generation.

Researchers both from academia and industry are invited to submit proposals for special sessions. A special session could consist of a set of individual presentations or a panel. Special sessions complement the regular technical program by highlighting emerging research topics. We welcome special sessions in all areas of Benchmarking and Optimizations (See https://www.benchcouncil.org/bench2023/cfp.html for topics). The participation of speakers from industry in a special session proposal is encouraged and will be positively evaluated when the proposal is assessed.

Proposal Submission

Please submit your proposals (maximum 3 pages) in PDF format by email to the Special Sessions Chair. Proposal should include the following information:

1. Title and Abstract: Propose a title and a brief abstract of no more than approximately 200 words that will allow conference attendees to understand the topic and the focus of the special session.

2. Rationale: Please explain why the topic of the special session is timely and compelling and why it is relevant to the Bench community.

3. Session Format: Please describe what the session format is (e.g., workshop, tutorial, panel, a series of invited talks, and so on).

4. Expected Length and/or Agenda: Please describe what the expected length of your session is (e.g., one hour, two hours, half day, and one day). Please feel free to include an expected agenda if needed.

5. Biographies: Please provide the list of organizers/speakers with a short bio, including the recent activity that is relevant to the topic of the special session.

6. Other Information: Please feel free to describe other important information about your proposed session. Note that the total proposal length should be within 3 pages.

* Proposals are due on or before August 31, 2023 11:59 pm (AoE) and should be submitted to the Special Sessions Chairs, Kai Shu and Reza Zafarani, by email ( kshu@iit.edu and/or reza@data.syr.edu ).

* Notification of acceptance will be sent to the organizers by September 15, 2023.

Calls for Papers

---

Full Papers: July 31, 2023, at 11:59 PM AoE

Notification: September 30, 2023, at 11:59 PM AoE

Final Papers Due: October 31, 2023, at 11:59 PM AoE

Conference Date: December 3-5, 2023

Venue: Sanya, China

Please note that citizens from up to 59 nations can visit Sanya without a Visa from the Chinese Government. Sanya is a beautiful seaside city, well known as Hawaii in China.

Submission website: https://bench2023.hotcrp.com/

---

The Bench conference encompasses a wide range of topics in benchmarks, datasets, metrics, indexes, measurement, evaluation, optimization, supporting methods and tools, and other best practices in computer science, medicine, finance, education, management, etc. Bench’s multidisciplinary and interdisciplinary emphasis provides an ideal environment for developers and researchers from different areas and communities to discuss practical and theoretical work. The topics of interest include, but are not limited to the following:

-- Benchmark science and engineering across multi-disciplines:The formulation of problems or challenges in emerging and future computing; The benchmarks, datasets, and indexes in multidisciplinary applications, e.g., medical, finance, education, management, psychology, etc; Benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges; Industry best practices.
-- Benchmark and standard specifications, implementations, and validations: Big Data, Artificial intelligence (AI), High performance computing (HPC), Machine learning, Big scientific data, Datacenter, Cloud, Warehouse-scale computing, Mobile robotics, Edge and fog computing, Internet of Things (IoT), Blockchain, Data management and storage, Financial, Education, Medical or other application domains.
-- Dataset: Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements; Analyses or meta-analyses of existing data and original articles on systems, technologies, and techniques that advance data sharing and reuse to support reproducible research; Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description; Tools that can generate large-scale data while preserving their original characteristics.
-- Workload characterization, quantitative measurement, design, and evaluation studies: Computer and communication networks, protocols and algorithms; Wireless, mobile, ad-hoc and sensor networks, IoT applications; Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks; HPC systems; Operating systems, file systems and databases; Virtualization, data centers, distributed and cloud computing, fog and edge computing; Mobile and personal computing systems; Energy-efficient computing systems; Real-time and fault-tolerant systems; Security and privacy of computing and networked systems; Software systems and services, and enterprise applications; Social networks, multimedia systems, web services; Cyber-physical systems, including the smart grid.
-- Methodologies, metrics, abstractions, algorithms, and tools: Analytical modeling techniques and model validation; Workload characterization and benchmarking; Performance, scalability, power and reliability analysis; Sustainability analysis and power management; System measurement, performance monitoring and forecasting; Anomaly detection, problem diagnosis and troubleshooting; Capacity planning, resource allocation, run time management and scheduling; Experimental design, statistical analysis, and simulation.
-- Measurement and evaluation:Measurement standards; Evaluation methodologies and metrics; Testbed methodologies and systems; Instrumentation, sampling, tracing and profiling of large-scale, real-world applications and systems; Collection and analysis of measurement data that yield new insights; Measurement-based modeling (e.g., workloads, scaling behavior, assessment of performance bottlenecks); Methods and tools to monitor and visualize measurement and evaluation data; Systems and algorithms that build on measurement-based findings; Advances in data collection, analysis and storage (e.g., anonymization, querying, sharing); Reappraisal of previous empirical measurements and measurement-based conclusions; Descriptions of challenges and future directions that the measurement and evaluation community should pursue.

Review Rules

---

1. The online discussion is blind. While the reviewers discuss the papers, they don’t know others’ identities beyond reviewer #A, #B, …. Hence, a single reviewer cannot easily assert seniority and silence other voices, or influence them beyond the strength of their arguments.
2. When the reviewers pointing out closeness to prior work that informs the reviewer’s decision to lower the novelty and contribution of a paper, they should provide a full citation to that prior work.
3. When the reviewers asking authors to draw a comparison with concurrent work (e.g., work that was published or appeared online *after* the paper submission deadline) or with preliminary work (e.g., a poster or abstract that is not archival), this comparison should not inform a lower score by the reviewer.
4. Provide useful and constructive feedback to the authors. Be respectful, professional and positive in your reviews and provide suggestions for the authors to improve their work.
5. Score the paper absolutely and relative to the group of papers you are reviewing.
Absolute overall merit - There are 4 grades you can give to each paper for absolute overall merit; the top 2 ratings mean that you think the paper is acceptable to the conference and the bottom 2 ratings mean that in your opinion the paper is below the threshold for the conference. Please assign these values thinking, whether the paper is above the threshold for the conference or below.
Relative overall merit – is based on the papers that you are reviewing. You can rank your papers and then group the papers into the 4 bins. Except for fractional errors, you should be dividing your papers equally into the 4 categories.
6. Reviewers must treat all submissions as strictly confidential and destroy all papers once the technical program has been finalized.
7. Reviewers must contact the PC chair or EIC if they feel there is an ethical violation of any sort (e.g., authors seeking support for a paper, authors seeking to identify who the reviewers are).
8. Do not actively look for author identities. Reviewers should judge a paper solely on its merits.
9. If you know the authors, do not publicize the authors. If you would like to recuse yourself from the review task, contact the PC Chair.
10. Reviewers should review the current submission. If you have reviewed a previous submission, make sure your review is based on the current submission.
11. Reviewers must not share the papers with students/colleagues.
12. Reviewers must compose the reviews themselves and provide unbiased reviews.
13. Do not solicit external reviews without consulting the PC chairs or EIC. If you regularly involve your students in the review process as part of their PhD training, contact the PC chairs. You are still responsible for the reviews. You may do this on no more than one of your reviews.
14. Reviewers must keep review discussions (including which papers you reviewed) confidential.
15. Do not discuss the content of a submitted paper/reviews with anyone other than officially on the submission management system like HotCRP or EasyChair during the online discussion period or the PC meeting (from now until paper publication in any venue).
16. Do not reveal the name of paper authors in case reviewers happen to be aware of author identity. (Author names of accepted papers will be revealed after the PC meeting; author names of rejected papers will never be revealed.)
17. Do not disclose the outcome of a paper until its authors are notified of its acceptance or rejection.
18. Do not download or acquire material from the review site that you do not need access to.
19. Do not disclose the content of reviews, including the reviewers' identities, or discussions about a paper.

Acknowledgments:
This set of review ethics is derived and based on the MICRO 2020, ASPLOS 2020-2021, ISCA 2020-21 review guidelines.

Submission Guidelines

---

Bench 2023 Submission website: https://bench2023.hotcrp.com/

Papers must be submitted in PDF. For a full paper, the page limit is 15 pages in the LNCS format, not including references. For a short paper, the page limit is 8 pages in the LNCS format, not including references. The review process follows a strict double-blind policy per the established Bench conference norms. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, the proceedings will be published by Springer LNCS (Pending, Indexed by EI). Please note that the LNCS format is the final one for publishing.

At least one author must pre-register for the symposium, and at least one author must attend the symposium to present the paper. Papers for which no author is pre-registered will be removed from the proceedings.

Formatting Instructions

LNCS latex template: https://www.benchcouncil.org/file/llncs2e.zip

Please make sure your submission satisfies ALL of the following requirements:

· All authors and affiliation information must be anonymized.
· Paper must be submitted in printable PDF format.
· Please number the pages of your submission.
· The submission must be formatted for black-and-white printers. Please make sure your figures are readable when printed in black and white.
· The submission must describe unpublished work that is not currently under review of any other conference or journal venues.

Bench 2023 Awards

---

Every year BenchCouncil presents the following awards at the Bench Conference.

• BenchCouncil Achievement Award ($3,000): This award recognizes a senior member who has made long-term contributions to benchmarking, measuring, and optimizing. The winner is eligible for the status of a BenchCouncil Fellow.


• BenchCouncil Rising Star Award ($1,000): This award recognizes a junior member who demonstrates outstanding potential for research and practice in benchmarking, measuring, and optimizing.


• BenchCouncil Best Paper Award ($1,000): This award recognizes a paper presented at the Bench conferences, which demonstrates potential impact on research and practice in benchmarking, measuring, and optimizing.


• BenchCouncil Distinguished Doctoral Dissertation Award ($2000): This award recognizes and encourages superior research and writing by doctoral candidates in the broad field of benchmarks, data, standards, evaluations, and optimizations community. This year, the award includes two tracks, including the BenchCouncil Distinguished Doctoral Dissertation Award in Computer Architecture ($1000) and BenchCouncil Distinguished Doctoral Dissertation Award in other areas ($1000).

Organizing Committee

---

General Co-Chairs:

Rakesh Agrawal, Data Insights Laboratories, San Jose, CA, USA

Aoying Zhou, East China Normal University

Program Co-Chairs:

Weining Qian, East China Normal University

Program Vice-Chair:

Biwei Xie, Institute of Computing Technology, CAS

Kai Shu, Illinois Institute of Technology

Web Co-chairs:

Jiahui Dai, BenchCouncil

---

Award Committees

---

2023 BenchCouncil Achievement Award Committee:

Prof. D. K. Panda, the Ohio State University

Prof. Lizy Kurian John, the University of Texas at Austin

Prof. Geoffrey Fox, Indiana University

Prof. Jianfeng Zhan, University of Chinese Academy of Sciences

Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020)

Prof. David J. Lilja, University of Minnesota, Minneapolis (Since 2021)

Prof. Jack J. Dongarra, University of Tennessee (Since 2022)

John L. Henning, Oracle (Since 2023)

2023 BenchCouncil Rising Star Award Committees:

Prof. D. K. Panda, the Ohio State University

Prof. Lizy Kurian John, the University of Texas at Austin

Prof. Geoffrey Fox, Indiana University

Prof. Jianfeng Zhan, University of Chinese Academy of Sciences

Prof. Torsten Hoefler, ETH Zürich (Since 2021)

Prof. Vijay Janapa Reddi, Harvard University (Since 2022)

Dr. Peter Mattson, Google, USA (Since 2022)

Dr. Wanling Gao , ICT, Chinese Academy of Sciences (pending)

Dr. Douwe Kiela, Stanford University (Since 2023)

2023 BenchCouncil Achievement Award Committees:

BenchCouncil Distinguished Doctoral Dissertation Award Committee in Other Areas:

Prof. Jack Dongarra, University of Tennessee

Dr. Xiaoyi Lu, The University of California, Merced

Dr. Jeyan Thiyagalingam, STFC-RAL

Dr. Lei Wang, ICT, Chinese Academy of Sciences

Dr. Spyros Blanas, The Ohio State University

BenchCouncil Distinguished Doctoral Dissertation Award Committee in Computer Architecture:

Prof. Resit Sendag, University of Rhode Island, USA

Dr. Peter Mattson, Google

Dr. Vijay Janapa Reddi, Harvard University

Dr. Wanling Gao, Chinese Academy of Sciences

---

Bench Steering Committees

---

Prof. Dr. Jack Dongarra, University of Tennessee

Prof. Dr. Geoffrey Fox, Indiana University

Prof. Dr. D. K. Panda, The Ohio State University

Prof. Dr. Felix, Wolf, TU Darmstadt

Prof. Dr. Xiaoyi Lu, University of California, Merced

Prof. Dr. Resit Sendag, University of Rhode Island, USA

Dr. Wanling Gao, ICT, Chinese Academy of Sciences & UCAS

Prof. Dr. Jianfeng Zhan, BenchCouncil

---