Jacob Rafati

Ph.D.
Electrical Engineering and Computer Science
Computational Cognitive Neuroscience Laboratry
Computational Optimization Group
Univeristy of California, Merced

I am on the job market and actively looking for a Research Scientist or Applied Reseach positions in machine learning, deep learning and reinforcement learning.

I have a Ph.D. in Electrical Engineering and Computer Science (EECS) from the University of California, Merced. In my Ph.D. dissertation, I have studied methods for learning representations in reinforcement learning. I have implemented methods for learning sparse representations in RL. I have also offered an approach for learning representations in Hierarchical Reinforcement Learning. I have also studied numerical optimization methods based on quasi-Newton methods in trust-region and line-search frameworks for large-scale machine learning. I have implemented scalable algorithms for solving the non-linear non-convex empirical risk minimization problems in deep learning and deep reinforcement learning applications. During my residency at the UC Merced, I have collaborated with Dr. David C. Noelle, Dr. Roummel F. Marcia and Dr. Jeff Yoshimi. See my publications for more information.

Education

University of California, Merced

Doctor of Philosophy
Electrical Engineering and Computer Science
2013 - 2019

Sharif Univeristy of Technology

Master of Science
Mechanical Engineering
2008 - 2010

Sharif Univeristy of Technology

Bachelor of Science
Mechanical Engineering
2003 - 2007

Publications

2019

[14] Jacob Rafati. (2019). Learning Representations in Reinforcement Learning. Ph.D. dissertation. University of California, Merced. USA.

[bibtex] [dissertation] [slides (PDF)] [slides (keynote slides)]

[13] Jacob Rafati, David C. Noelle. (2019). Unsupervised Subgoal Discovery Method for Learning Hierarchical Representations. In 7th International Conference on Learning Representations, ICLR 2019 Workshop on "Structure & Priors in Reinforcement Learning", New Orleans, LA, USA.

[bibtex] [paper] [code] [slides] [poster] [project website]

[12] Jacob Rafati, David C. Noelle. (2019). Unsupervised Methods For Subgoal Discovery During Intrinsic Motivation in Model-Free Hierarchical Reinforcement Learning. In 33rd AAAI Conference on Artificial Intelligence (AAAI-19). Workshop on Knowledge Extraction From Games. Honolulu, Hawaii. USA.

[bibtex] [paper] [slides] [proceeding] [ceurws-paper] [keg19-paper] [keg19-list] [code] [project website]

[11] Jacob Rafati, and David C. Noelle (2019). Learning Representations in Model-Free Hierarchical Reinforcement Learning. In 33rd AAAI Conference on Artificial Intelligence (AAAI-19), Honolulu, Hawaii.

[bibtex] [abstract] [poster] [long version preprint]

[10] Jacob Rafati, and Roummel F. Marcia (2019). Deep Reinforcement Learning via L-BFGS Optimization. arXiv e-print (arXiv:1811.02693).

[bibtex] [preprint]

[9] Jacob Rafati, and David C. Noelle (2019). Learning Representations in Model-Free Hierarchical Reinforcement Learning. arXiv e-print (arXiv:1810.10096).

[bibtex] [preprint]

2018

[8] Jacob Rafati, and Roummel F. Marcia (2018). Improving L-BFGS Initialization For Trust-Region Methods In Deep Learning. In 17th IEEE International Conference on Machine Learning and Applications (ICMLA 2018), Orlando, Florida.

[bibtex] [paper] [proceeding] [ieee-abstract] [doi] [sildes] [project website] [code]

[7] Jacob Rafati, Omar DeGuchy and Roummel F. Marcia (2018). Trust-Region Minimization Algorithm for Training Responses (TRMinATR): The Rise of Machine Learning Techniques. In 26th European Signal Processing Conference, Rome, Italy.

[bibtex] [paper] [proceeding] [ieee-abstract] [doi] [project website] [code]

2017

[6] Jacob Rafati and David C. Noelle (2017). Sparse Coding of Learned State Representations in Reinforcement Learning. In Cognitive Computational Neuroscience Conference, Newyork City, NY.

[bibtex] [abstract] [poster] [ link ] [project website] [code]

2015

[5] Jacob Rafati and David C. Noelle (2015). Lateral Inhibition Overcomes Limits of Temporal Difference Learning. In 37th Annual Cognitive Science Society Meeting, Pasadena, CA, USA.

[bibtex] [paper] [poster] [link] [project website] [code]

2014

[4] Jacob Rafati, Mohsen Asghari and Sachin Goyal (2014). Effects of DNA Encapsulation On Buckling Instability of Carbon Nanotube based on Nonlocal Elasticity Theory. In Proceedings of the ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE 2014), Buffalo, New York, USA.

[bibtex] [paper] [doi] [link]

2013

[3] Mohsen Asghari, Jacob Rafati and Reza Naghdabadi (2013). Torsional instability of carbon nano-peapods based on the nonlocal elastic shell theory. Physica E: Low-dimensional Systems and Nanostructures, 47:316-323.

[bibtex] [paper] [doi] [link]

2011

[2] Mohsen Asghari, Reza Naghdabadi and Jacob Rafati (2011). Small scale effects on the stability of carbon nano-peapods under radial pressure. Physica E: Low-dimensional Systems and Nanostructures, 43(5):1050-1055.

[bibtex] [paper] [doi] [link]

2010

[1] Mohsen Asghari and Jacob Rafati (2010). Variational Principles for the Stability Analysis of Multi-Walled Carbon Nanotubes Based on a Nonlocal Elastic Shell Model. In 10th ASME Biennial Conference on Engineering Systems Design and Analysis, paper no. ESDA2010-24473, pages 591-598.

[bibtex] [paper] [doi] [link]

News

2019

April, 26th, 2019. I have succesfully defended my Ph.D. dissertation. The title of my dissertation is "Learning Representation in Reinforcement Learning". The Ph.D. commitee members are Dr. David. C. Noelle, Dr. Roummel F. Marcia, Dr. Shawn Newsam, Dr. Marcelo Kallmann and Dr. Jeffrey Yoshimi.

May 2019. I will present our work on "Unsupervised Subgoal Discovery Method for Learning Hierarchical Representations" on Monday, May 6th, 2019 in ICLR 2019 SPiRL workshop at Room R4, Ernest N. Morial Convention Center, New Orleans. For more details, see http://spirl.info/2019/program/.

April 2019. I will give a talk on "Quasi-Newton Optimization For Large-scale Machine Learning" on Saturday, April, 27th, 2019 in Southern California Applied Mathematics Symposium (SOCAMS 2019) at the California Institute of Technology. For more details, see http://cmx.caltech.edu/socams/. To read the submitted abstract, click here.

April 2019. I will defend my Ph.D. dissertation on Friday, April, 26th, 2019 from 1:00PM to 2:00PM. The defense will be held in Half Dome Conference Room located at the UC Merced Social Science and Management, SSM 317. For more details, see this flyer.

March 2019. The paper, "Unsupervised Subgoal Discovery Method for Learning Hierarchical Representations", co-authored with David C. Noelle has been accepted for presentation at the ICLR 2019 Workshop on "Structure & Priors in Reinforcement Learning" (SPiRL). Congratulations David!

January 2019. I gave a talk on our paper "Unsupervised Methods For Subgoal Discovery During Intrinsic Motivation in Model-Free Hierarchical Reinforcement Learning", co-authored with David C. Noelle, in AAAI 2019 Workshop on Knowledge Extraction in Games (KEG 2019) in Honolulu, Hawaii.

January 2019. I have been awarded the Graduate Dean's Dissertation Fellowship ($11,000) to support writing my Ph.D. dissertation in Spring 2019 Semester. I would like to thank Dean Zatz for the awrad, Dr. David C. Neolle and and Dr. Marcelo Kallmann for the nomination and support.

January 2019. The paper, "Learning Representations in Model-Free Hierarchical Reinforcement Learning" co-authored with Dr. David C. Noelle has been selected for poster presentation in 33rd AAAI Conference on Artificial Intelligence (AAAI-19) in Honolulu, Hawaii for Student Abstract and Poster Program. Congratulations David!

2018

December 2018. The paper, "Improving L-BFGS Initialization For Trust-Region Methods In Deep Learning" co-authored with Dr. Roummel F. Marcia has been accpeted to the 17th IEEE International Conference on Machine Learning and Applications conference (ICMLA 2018) in Orlando, Florida for oral presentation. Congratulations Roummel!

November 2018. I will give a talk in UC Merced EECS Graduate Seminar on November 2nd, 2018 in COB 265 from 12:00 to 1:15 PM. Title: Limited-Memory Quasi-Netwon Optimization Methods in Deep Learning.

August 2018. I am officially a Ph.D. Candidate. Dr. Marjorie S. Zatz, the vice provost and Dean of the Graduate Division congratulated me the advancement to the Ph.D. candidacy.

June 2018. I passed the qualification examination by defending my Ph.D. dissertation research proposal succesfully. The title of my dissertation is "Learning Representation in Reinforcement Learning". The Ph.D. commitee members are Dr. David. C. Noelle, Dr. Roummel F. Marcia, Dr. Shawn Newsam, Dr. Marcelo Kallmann and Dr. Jeffrey Yoshimi.

May 2018. The paper, "Trust-Region Minimization Algorithm for Training Responses (TRMinATR): The Rise of Machine Learning Techniques" has been accepted to the 26th European Signal Processing Conference (EUSIPCO 2018), Italy, Rome. Co-authored with Omar DeGuchy and Dr. Roummel F. Marcia.

Experience

Ph.D. Researcher

University of California, Merced
Electrical Engineering and Computer Science
Computational Cognitive Neuroscience Laboratory
  • Methoods for learning sparse representation in reinforcement learning.
  • Methods for learning representations in model-free hierarchical reinforcement learning.
  • Unsupervised Subgoal Discovery in model-free hierarchical reinforcement learning.
Computational Optimization Group
  • Quasi-Newton Trust-region Optimization Methoods for Deep Learning.
  • Quasi-Newton Line-search Optimization Methoods for Deep Learning.
  • Methods for Initialization of Quasi-Newton Matrices.
  • Deep Reinforcement Learning via Quasi-Newton Optimization.
2013 - 2019

Teaching Assistant

Univerisity of California, Merced
School of Engineering
List of Courses:
  • Introduction to Artificial Intelligence (Java), Fall 2017, Fall 2018.
  • Computational Coginitive Neuroscience (Emergent), Spring 2017, Spring 2018.
  • Computer Organization (C and MIPS Assembly), Spring 2016, Summer 2018.
  • Introduction to Computing I (Java), Spring 2015,Fall 2014. Summer 2018.
  • Engineering Programming (Matlab and Fortran), Fall 2013.
  • Finite Element Analysis, (MatLab and ComSol Lab) Spring 2014.
2013 - 2018

Iran Power Development Company

Office of Engineering
  • Design, Engineering, Construction Management of 7 Power-plants projects (5200 MW).
  • Project Management on a Research Collaboration with Academia on Computational Methods for Designing Power-plant's Cooling Towers.
2009 - 2012

M.Sc. Researcher

Mechanical Engineering
  • Computational algorithms to study large-scale nano-structures under mechanical loadings.
  • Applied Mathematics, nonlinear elasticity, nonlocal elasticity.
  • Computational Algorithms for Large-scale Mechcanical Systems.
2008 - 2010

Skills

Programming Languages
  • Python
  • Matlab
  • Java
  • Bash
  • Javascript
  • C++
Machine Learning Libraries
  • TensorFlow and Keras
  • Pytorch
  • SciPy and Numpy
Workflow
  • Multi-GPU and CPU clusters parallel comupations
  • Cross Platform Pipelining
  • Interdisciplinary and Cross Functional Teams

Interests

Research Interests
  • Machine Learning
  • Large-scale Numerical Optimization
  • Deep Learning
  • Reinforcement Learning
  • Applied Mathematics
  • Dynamical Systems and Control
  • Artificial Intelligence
  • Game Theory
Other Interests

Apart from being my research in computer science, I have been active as an artist too. My art includes, performeronce art, paintings, installations, video, and interactive arts. You can visit my art website at http://jacobrafati.com for more details.

I enjoy walking in nature. I love coffee and coffeeshops. I mostly lived in cities, Tabriz, Tehran and San Francisco. I like Poker and mind games. I love adventurous sports such as Bungee jumping. I can speak fulently in 3 languages, English, Azeri and Farsi.

Awards & Certifications

  • Graduate Dean's Dissertation Fellowship. Spring 2019
  • AAAI 2019 travel fund. January 2019.
  • Graduate Excel Mentorship Program Fellowship. Fall 2018
  • Bobcat EECS Fellowships. 2014 -- 2018
  • UC Merced EECS Travel Fund 2014
  • UC Merced ME Travel Fund 2014
  • National grant for contribution in the field of Nanotechnology
  • 141th Place in the Iranian National University Entrance Exam among 350,000 Participants
  • 1st Place in Entrance Exam for National Organization for Development of Exceptional Talents (NODET)