Jacob Rafati

Ph.D. of Electrical Engineering and Computer Science
Univeristy of California, Merced

I am currently an applied researcher at Expedia Group, where I develop machine learning algorithms for complex problems in marketing and advertising domains such as real-time bidding in Google Hotel Ads. I have a Ph.D. in Electrical Engineering and Computer Science (EECS) from the University of California, Merced. In my Ph.D. dissertation, I studied methods for learning representations in reinforcement learning as well as numerical optimization methods based on quasi-Newton approach for deep learning and deep reinforcement learning applications. See my publications for more information.

Education

University of California, Merced

Doctor of Philosophy
Electrical Engineering and Computer Science
2013 - 2019

Sharif Univeristy of Technology

Master of Science
Mechanical Engineering
2008 - 2010

Sharif Univeristy of Technology

Bachelor of Science
Mechanical Engineering
2003 - 2007

Publications

2020

[17] Jacob Rafati, and Roummel F. Marcia (2020). Quasi-Newton Optimization Methods For Deep Learning Applications. Published as a chapter in Deep Learning Applications book, Springer.

[bibtex] [pdf] [abstract] [springer]

2019

[16] Jacob Rafati, and David C. Noelle (2019). Efficient Exploration through Intrinsic Motivation Learning for Unsupervised Subgoal Discovery in Model-Free Hierarchical Reinforcement Learning. arXiv e-print (arXiv:1911.10164).

[bibtex] [pdf] [abstract]

[15] Jacob Rafati, and David C. Noelle (2019). Learning sparse representations in reinforcement learning. arXiv e-print (arXiv:1909.01575). Under Review.

[bibtex] [pdf] [abstract]

[14] Jacob Rafati. (2019). Learning Representations in Reinforcement Learning. Ph.D. dissertation. University of California, Merced. USA.

[bibtex] [pdf] [UC-open-access] [slides (pdf)] [slides (keynote)]

[13] Jacob Rafati, David C. Noelle. (2019). Unsupervised Subgoal Discovery Method for Learning Hierarchical Representations. In 7th International Conference on Learning Representations, ICLR 2019 Workshop on "Structure & Priors in Reinforcement Learning", New Orleans, LA, USA.

[bibtex] [pdf] [code] [slides] [poster] [project website]

[12] Jacob Rafati, David C. Noelle. (2019). Unsupervised Methods For Subgoal Discovery During Intrinsic Motivation in Model-Free Hierarchical Reinforcement Learning. In 33rd AAAI Conference on Artificial Intelligence (AAAI-19). Workshop on Knowledge Extraction From Games. Honolulu, Hawaii. USA.

[bibtex] [pdf] [slides] [proceeding] [ceurws-paper] [keg19-paper] [keg19-list] [code] [project website]

[11] Jacob Rafati, and David C. Noelle (2019). Learning Representations in Model-Free Hierarchical Reinforcement Learning. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI-19). 33(01), 10009-10010. Honolulu, Hawaii.

[bibtex] [pdf] [poster] [proceeding] [doi] [supplementary]

[10] Jacob Rafati, and Roummel F. Marcia (2019). Deep Reinforcement Learning via L-BFGS Optimization. arXiv e-print (arXiv:1811.02693).

[bibtex] [pdf] [abstract]

[9] Jacob Rafati, and David C. Noelle (2019). Learning Representations in Model-Free Hierarchical Reinforcement Learning. arXiv e-print (arXiv:1810.10096).

[bibtex] [pdf] [abstract]

2018

[8] Jacob Rafati, and Roummel F. Marcia (2018). Improving L-BFGS Initialization For Trust-Region Methods In Deep Learning. In 17th IEEE International Conference on Machine Learning and Applications (ICMLA 2018), Orlando, Florida.

[bibtex] [pdf] [proceeding] [ieee-abstract] [doi] [sildes] [project website] [code]

[7] Jacob Rafati, Omar DeGuchy and Roummel F. Marcia (2018). Trust-Region Minimization Algorithm for Training Responses (TRMinATR): The Rise of Machine Learning Techniques. In 26th European Signal Processing Conference, Rome, Italy.

[bibtex] [pdf] [proceeding] [ieee-abstract] [doi] [project website] [code]

2017

[6] Jacob Rafati and David C. Noelle (2017). Sparse Coding of Learned State Representations in Reinforcement Learning. In Cognitive Computational Neuroscience Conference, Newyork City, NY.

[bibtex] [pdf] [poster] [ link ] [project website] [code]

2015

[5] Jacob Rafati and David C. Noelle (2015). Lateral Inhibition Overcomes Limits of Temporal Difference Learning. In 37th Annual Cognitive Science Society Meeting, Pasadena, CA, USA.

[bibtex] [pdf] [poster] [link] [project website] [code]

2014

[4] Jacob Rafati, Mohsen Asghari and Sachin Goyal (2014). Effects of DNA Encapsulation On Buckling Instability of Carbon Nanotube based on Nonlocal Elasticity Theory. In Proceedings of the ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE 2014), Buffalo, New York, USA.

[bibtex] [pdf] [doi] [link]

2013

[3] Mohsen Asghari, Jacob Rafati and Reza Naghdabadi (2013). Torsional instability of carbon nano-peapods based on the nonlocal elastic shell theory. Physica E: Low-dimensional Systems and Nanostructures, 47:316-323.

[bibtex] [pdf] [doi] [link]

2011

[2] Mohsen Asghari, Reza Naghdabadi and Jacob Rafati (2011). Small scale effects on the stability of carbon nano-peapods under radial pressure. Physica E: Low-dimensional Systems and Nanostructures, 43(5):1050-1055.

[bibtex] [pdf] [doi] [link]

2010

[1] Mohsen Asghari and Jacob Rafati (2010). Variational Principles for the Stability Analysis of Multi-Walled Carbon Nanotubes Based on a Nonlocal Elastic Shell Model. In 10th ASME Biennial Conference on Engineering Systems Design and Analysis, paper no. ESDA2010-24473, pages 591-598.

[bibtex] [pdf] [doi] [link]

[0] Jacob Rafati (2010). Stability Analysis of Hybrid Nanotubes based on the Nonlocal Continuum Theories. M.Sc. Thesis. Sharif University of Technology. Tehran, Iran.

[bibtex] [pdf (in farsi)] [link (sharif library)]

Experience

Applied Researcher

Global Marketing Organization, Expedia Group
  • Computational Advertising and Marketing.
  • Methods based on Supervised Learning, Unsupervised Learning and Reinforcement Learning for Real-Time Bidding in Google Hotel Ads.
  • Joint optimization of predicting Users' behavior, forcasting market's price and optimizing bid for each hotel, point of sale, device, booking dates, etc.
July 2019 - current

Ph.D. Researcher

University of California, Merced
Electrical Engineering and Computer Science
Computational Cognitive Neuroscience Laboratory
  • Methods for learning sparse representation in reinforcement learning.
  • Methods for learning representations in model-free hierarchical reinforcement learning.
  • Unsupervised Subgoal Discovery in model-free hierarchical reinforcement learning.
Computational Optimization Group
  • Quasi-Newton Trust-region Optimization Methoods for Deep Learning.
  • Quasi-Newton Line-search Optimization Methoods for Deep Learning.
  • Methods for Initialization of Quasi-Newton Matrices.
  • Deep Reinforcement Learning via Quasi-Newton Optimization.
2013 - 2019

Teaching Assistant

Univerisity of California, Merced
School of Engineering
List of Courses:
  • Introduction to Artificial Intelligence (Java), Fall 2017, Fall 2018.
  • Computational Coginitive Neuroscience (Emergent), Spring 2017, Spring 2018.
  • Computer Organization (C and MIPS Assembly), Spring 2016, Summer 2018.
  • Introduction to Computing I (Java), Spring 2015,Fall 2014. Summer 2018.
  • Engineering Programming (Matlab and Fortran), Fall 2013.
  • Finite Element Analysis, (MatLab and ComSol Lab) Spring 2014.
2013 - 2018

Iran Power Development Company

Office of Engineering
  • Design, Engineering, Construction Management of 7 Power-plants projects (5200 MW).
  • Project Management on a Research Collaboration with Academia on Computational Methods for Designing Power-plant's Cooling Towers.
2009 - 2012

M.Sc. Researcher

Mechanical Engineering
  • Computational algorithms to study large-scale nano-structures under mechanical loadings.
  • Applied Mathematics, nonlinear elasticity, nonlocal elasticity.
  • Computational Algorithms for Large-scale Mechcanical Systems.
2008 - 2010

Interests

Research Interests
  • Machine Learning
  • Large-scale Numerical Optimization
  • Deep Learning
  • Reinforcement Learning
  • Applied Mathematics
  • Dynamical Systems and Control
  • Artificial Intelligence
  • Game Theory
Other Interests

Apart from being my research in computer science, I have been active as an artist too. My art includes, performeronce art, paintings, installations, video, and interactive arts. You can visit my art website at http://jacobrafati.com for more details.

I enjoy walking in nature. I love coffee and coffeeshops. I mostly lived in cities, Tabriz, Tehran, San Francisco and Seattle. I enjoy playing Poker and mind games. I love adventurous sports such as Bungee jumping. I can speak fulently in 3 languages, English, Azeri and Farsi.

Awards & Certifications

  • Graduate Dean's Dissertation Fellowship. Spring 2019
  • AAAI 2019 travel fund. January 2019.
  • Graduate Excel Mentorship Program Fellowship. Fall 2018
  • Bobcat EECS Fellowships. 2014 -- 2018
  • UC Merced EECS Travel Fund 2014
  • UC Merced ME Travel Fund 2014
  • National grant for contribution in the field of Nanotechnology
  • 141th Place in the Iranian National University Entrance Exam among 350,000 Participants
  • 1st Place in Entrance Exam for National Organization for Development of Exceptional Talents (NODET)