The Bench conference encompasses a wide range of topics in benchmarks, datasets, metrics, indexes, measurement, evaluation, optimization, supporting methods and tools, and other best practices in computer science, medicine, finance, education, management, etc. Bench’s multidisciplinary and interdisciplinary emphasis provides an ideal environment for developers and researchers from different areas and communities to discuss practical and theoretical work. The topics of interest include, but are not limited to the following:
Benchmark science and engineering across multi-disciplines:
- The formulation of problems or challenges in emerging and future computing;
- The benchmarks, datasets, and indexes in multidisciplinary applications, e.g., medical, finance, education, management, psychology, etc;
- Benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges; Industry best practices.
Benchmark and standard specifications, implementations, and validations:
- Big Data, Artificial intelligence (AI), High performance computing (HPC), Machine learning, Big scientific data, Datacenter, Cloud, Warehouse-scale computing, Mobile robotics, Edge and fog computing, Internet of Things (IoT), Blockchain, Data management and storage, Financial, Education, Medical or other application domains.
- Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements;
- Analyses or meta-analyses of existing data and original articles on systems, technologies, and techniques that advance data sharing and reuse to support reproducible research;
- Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description;
- Tools that can generate large-scale data while preserving their original characteristics.
Methodologies, metrics, abstractions, algorithms, and tools
- Analytical modeling techniques and model validation;
- Workload characterization and benchmarking;
- Performance, scalability, power and reliability analysis;
- Sustainability analysis and power management;
- System measurement, performance monitoring and forecasting;
- Anomaly detection, problem diagnosis and troubleshooting;
- Capacity planning, resource allocation, run time management and scheduling;
- Experimental design, statistical analysis, and simulation.
Workload characterization, quantitative measurement, design, and evaluation studies
- Computer and communication networks, protocols and algorithms;
- Wireless, mobile, ad-hoc and sensor networks, IoT applications;
- Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks;
- HPC systems; Operating systems, file systems and databases; Virtualization, data centers, distributed and cloud computing, fog and edge computing;
- Mobile and personal computing systems, energy-efficient computing systems; real-time and fault-tolerant systems, security and privacy of computing and networked systems, software systems and services, and enterprise applications, social networks, multimedia systems, web services, cyber-physical systems, including the smart grid.
Measurement and evaluation
- Measurement standards, evaluation methodologies and metrics, testbed methodologies and systems;
- Instrumentation, sampling, tracing and profiling of large-scale, real-world applications and systems;
- Measurement-based modeling (e.g., workloads, scaling behavior, assessment of performance bottlenecks); methods and tools to monitor and visualize measurement and evaluation data;
- Systems and algorithms that build on measurement-based findings; advances in data collection, analysis and storage (e.g., anonymization, querying, sharing); Reappraisal of previous empirical measurements and measurement-based conclusions;
- Descriptions of challenges and future directions that the measurement and evaluation community should pursue.