Lijie Xu (许利杰)

Assistant Research Professor
Technology Center of Software Engineering (TCSE)
Institute of Software, Chinese Academy of Sciences (ISCAS)

Email: xulijie AT otcaix DOT iscas DOT ac DOT cn

About Me

My research interests focus on big data systems/applications. Currently, I'm working on system reliability, system scalability, memory management techniques, and stream processing.

I got my PhD Degree from Institute of Software, Chinese Academy of Sciences in Jan. 2016 and my Bachelor's Degree from Wuhan University in 2009.

  1. Lijie Xu, Wensheng Dou, Feng Zhu, Chushu Gao, Jie Liu, and Jun Wei. Characterizing and Diagnosing Out of Memory Errors in MapReduce Applications. The Journal of Systems and Software (JSS), 2017. [pdf]
  2. Lijie Xu, Wensheng Dou, Feng Zhu, Chushu Gao, Jie Liu, Hua Zhong, Jun Wei. A Characteristic Study on Out of Memory Errors in Distributed Data-Parallel Applications. In the 26th IEEE International Symposium on Software Reliability Engineering (ISSRE 2015), Washington DC, USA, Nov. 2015. [pdf][OOM Cases]
  3. Lijie Xu, Jie Liu, and Jun Wei. FMEM: A Fine-grained Memory Estimator for MapReduce Jobs. In Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013), pages 65-68, San Jose, USA, June 2013. [pdf]
  4. Lijie Xu, Jie Liu, and Jun Wei. MapReduce Framework Optimization via Performance Modeling. In Proceedings of the 26th IEEE International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPS PhD Forum 2012), pages 2506-2509, Shanghai, China, May 2012. [pdf]
  5. Yingying Zheng, Lijie Xu, Wei Wang, Wei Zhou, Ying Ding. A Reliability Benchmark for Big Data Systems on JointCloud. In the Eighth International Workshop on Joint Cloud Computing (JCC 2017) in conjunction with the 37th International Conference on Distributed Computing Systems (ICDCS 2017), Atlanta, USA, Jun. 2017. [pdf]
  6. Feng Zhu, Jie Liu, Sa Wang, Jiwei Xu, Lijie Xu, Jixin Ren, Dan Ye, Jun Wei, Tao Huang. Hug the Elephant: Migrating a Legacy Data Analytics Application to Hadoop Ecosystem. In Proceedings of the 32th IEEE International Conference on Software Maintenance and Evolution (ICSME 2016), pages 178-187, Raleigh, North Carolina, USA, Oct. 2016. [pdf]
  7. Feng Zhu, Jie Liu, Lijie Xu, Dan Ye, Jun Wei, Tao Huang. A Lightweight Evaluation Framework for Table Layouts in MapReduce Based Query Systems. In Proceedings of the 17th Asia-Pacific Web Conference (APWeb 2015), pages 473-484, GuangZhou, China, Sept. 2015. [pdf]
  8. Feng Zhu, Jie Liu, Lijie Xu. A Fast and High Throughput SQL Query System for Big Data. In Proceedings of the 13th International Conference on Web Information Systems Engineering (WISE 2012), pages 783-788, Paphos, Cyprus, Nov. 2012. [pdf]

  1. Intern, Taobao technology department, Alibaba (Nov. 2014 - Feb. 2015)
    1. Improve and optimize Spark, fix a critical bug SPARK-4672 (elected as an important update of Spark 1.2.0)
    2. Analyze parameter server systems, including Petuum [My notes]
  2. Intern, System Research Group, Microsoft Research Asia (Apr. 2013 - Sept. 2013)
    1. Design and implement a RPC prototype to ease concurrent/asynchronous/non-deterministic programming
    2. Research on memory management problems in MapReduce applications
  3. Intern, Data Mining (NLP) Group, Tencent (June 2010 - Aug. 2010)
    1. Mining synonyms from dictionaries and Wikipedia
    2. Mined 250K+ pairs of commonly used synonyms
    3. Mined 200K+ pairs of entity name synonyms
  4. Intern, Institute of Computing Technology, CAS (June 2008 - Aug. 2008)
    Work on workflow-scheduling algorithms in grid computing

Technical Reports
  1. Spark Internals (talking about the design and implementation of Apache Spark, with 1,800+ Stars and 800+ Forks in github), 2014-2015
  2. Machine Learning Notes (in Chinese, with 100,000+ pageviews), 2012
  3. Hadoop Memory Usage Model, 2013

  1. Out of memory analysis for big data applications
  2. Memory usage modeling for big data applications
  3. Tools for diagnosing memory problems in big data applications

Contributions to Apache Hadoop/Spark
  1. Fix the bug SPARK-4672 (Iterative Spark jobs may suffer from StackOverflow errors)
  2. Fix the bug MAPREDUCE-4882 (The wrong estimating size of output file causes disk full error in spill phase)
  3. Fix the bug MAPREDUCE-4883 (The improper framework buffer size affects jobs' performance)


  1. Reviewer of Fast Data Processing with Spark – Second Edition, PACKT Publishing, 2015.
  2. Reviewer of Mastering Apache Spark, PACKT Publishing, 2015.
  3. One of the translators of Stanford Deep Learning Tutorial, 2013

  1. Play acoustic/classical guitar
  2. Swimming