LLM Standards Working Group

  1. Home
  2. LLM Standards Working Group

Goal

The LLM Standards Working Group aims to enhance technical standardization and normalization, drive technological innovation and interdisciplinary integration, and ensure the extensive, safe, and efficient application of LLM in various social domains. By establishing a world-leading technical standard system, it promotes the development of LLM technology and industry globally, laying the foundation for future alliances in general artificial intelligence technology.

Vision

Visioning a future where standardization efforts pave the way for the extensive adoption of LLM technology across domains like education, healthcare, finance, and intelligent manufacturing, we aspire to transform these advancements into a cornerstone for industrial upgrading, augmented social governance, and enhanced quality of life for all, ultimately realizing the vision of general artificial intelligence.

Committees

Leader of the Working Group:

  • Jianfeng Zhan, Founding Chair, The International Open Benchmark Council

Deputy Leaders of the Working Group:

  • Astronomy and Space Field
    • Ziming Zou, Deputy Director & Professor, National Space Science Center, Chinese Academy of Sciences
    • Zujie Ren, Professor, Zhejiang Lab
  • Materials Science Field
    • Yue Liu, Professor, Shanghai University
  • Other Scientific LLM Fields
    • Yuanchun Zhou, Deputy Director & Professor, Computer Network Information Center, Chinese Academy of Sciences
  • Sovereign LLM Field
    • Yunquan Zhang, Professor, Institute of Computing Technology, Chinese Academy of Sciences
    • Cheng Qian, Professor, Guangdong Institute of Intelligence Science and Technology
  • Ethics Field
    • Zhaofeng He, Professor, Beijing University of Posts and Telecommunications
  • Systems and Chips Field
    • Large-scale Code Generation Subfield
      • Yanjun Wu, Deputy Director & Professor, Institute of Software, Chinese Academy of Sciences
    • AI Inference Systems Subfield
      • Jason Jiaļ¼ŒAmazon
      • Li Ruan, Associate Professor, Beihang University
    • AI Systems Subfield
      • Jidong Zhai, Professor, Tsinghua University
      • Gang Lu, Alibaba
    • GPGPU Subfield
      • Xiaoyao Liang, Professor, Shanghai Jiao Tong University
    • Distributed AI Subfield
      • Saiqin Long, Professor, Jinan University
      • Pengfei Chen, Sun Yat-sen University
    • LLM Terminals Subfield
      • Rui Zhang, China Academy of Information and Communications Technology
  • Foundation Model Field
    • Wanling Gao, Associate Professor, Institute of Computing Technology, Chinese Academy of Sciences
    • Kunlin Zhan, AI Technical Director, 58.com
  • Education Field
    • Bin Cui, Professor, Peking University
    • Xiaoliang Xiong, Associate Professor, Peking University
  • Psychology Field
    • Tingshao Zhu, Professor, Institute of Psychology, Chinese Academy of Sciences
  • Medical Field
    • Pharmaceutical Subfield
      • Yi Zhao, Professor, Institute of Computing Technology, Chinese Academy of Sciences
    • Critical Care Subfield
      • Zhifei Zhang, Associate Professor, Capital Medical University
    • Ophthalmology Subfield
      • Yanwu Xu, Professor, South China University of Technology
    • Cancer Immunotherapy Subfield
      • Yunyou Huang, Guangxi Normal University
  • Industrial Field
    • Energy Efficiency Subfield
      • Weiwei Lin, Professor, South China University of Technology
      • Chunjie Luo, Associate Professor, Institute of Computing Technology, Chinese Academy of Sciences
    • Smart City Subfield
      • Feng Yuan, Professor & Founder, Zhongke Zhicheng (Guangzhou) Information Technology Co., Ltd.
    • Communications Subfield
      • Feng Wang, Institute of Big Data and Artificial Intelligence, China Telecom Research Institute

Current Members

  • Institute of Software, Chinese Academy of Sciences
  • Airt.ai
  • Huawei
  • Institute of Computing Technology, Chinese Academy of Sciences
  • Fudan University
  • Alibaba
  • Shanghai Jiao Tong University
  • Tencent
  • Tsinghua University
  • National University of Defense Technology
  • Northeastern University
  • Baidu
  • Guizhou University
  • Wuhan University
  • Hubei University
  • China Electronics Standardization Institute
  • Peking University
  • Sun Yat-sen University
  • South China University of Technology
  • Jinan University
  • Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
  • Zhejiang Lab
  • Shanghai University
  • Peng Cheng Laboratory
  • Amazon
  • Zhongguancun Laboratory
  • National Institute of Metrology, China
  • China Software Testing Center
  • China Academy of Information and Communications Technology
  • Xiaomi
  • Sugon
  • Inspur
  • 58.com
  • BaiChuan Intelligence
  • Zhongke Zhicheng (Guangzhou) Information Technology Co., Ltd.
  • Guangxi Normal University
  • Capital Medical University
  • Institute of Psychology, Chinese Academy of Sciences

Rules of Procedure

Process for Submitting Standards

  • Prepare and document
  • Feedback from Experts
  • Audit and certification
  • Publish and renew
  • Abolish and renew

Quality Management System

  • First Step: Develop a quality management system (QMS) unique to the business.
  • Second Step: Identity Core business processes, determine the improvement or advantage.
  • Third Step: Create a document to outline the process at all levels of business.
  • Fourth Step: Organize a committee of experts to approve.
  • Fifth Step: Implement the QMS and Schedule an external audit
  • Sixth Step: Review and Issue

Work Plan for Standards

  • General LLM
  • Astronomy and Space LLM
  • Materials Science LLM
  • Sovereign LLM
  • Large-scale Code Generation
  • AI Inference System
  • Medical LLM
  • Industrial LLM
  • Power Consumption LLM
  • Formulation of Basic Standards for LLM
  • Customization Specifications for LLM in Vertical Domains
  • Standard Processes for Data Annotation and Preprocessing
  • Technical Standards for LLM Training and Fine-tuning
  • Evaluation Science and Engineering of LLM
  • Security and Privacy Protection Standards for LLM
  • Domain-specific Standardization Implementation for LLM
  • Continuous Monitoring and Improvement Mechanisms for LLM Standards and Specifications

Meeting Minutes

Meeting Minutes of the First Work Group Meeting - December 5, 2024