Clark School Home UMD
ECE

ECE News Story

ECE Faculty and Alumni Co-Author Book on Markov Decision Processes

ECE Faculty and Alumni Co-Author Book on Markov Decision Processes


A new book titled Simulation-based Algorithms for Markov Decision Processes was co-written by four authors with ties to the Electrical and Computer Engineering Department: former ECE Chair and current Professor Steve Marcus (ECE/ISR); Professor Michael Fu (BUS/ECE/ISR); former Institute for Systems Research (ISR) postdoctoral researcher Hyeong Soo Chang, who now teaches at Sogang University in Seoul, Korea; and ECE alumnus Jiaqiao Hu, who was advised by Marcus and Fu and is now an assistant professor at SUNY Stony Brook University.

The new publication, part of the Springer's Communications and Control Engineering series, explores Markov decision processes (MDP). MDPs are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. The book helps provide sampling and population-based numerical algorithms to overcome the difficulties of computing an optimal solution using MDP models in terms of a policy or value function.

Specific approaches used in the book include multi-stage adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search. The book provides specific algorithms, illustrative numerical examples and rigorous theoretical convergence results. The new book should appeal not only to researchers in MDPs, stochastic modeling and control, and simulation but should also be a valuable reference for students of control and operations research.

For more information about the book, visit the Springer website.

February 26, 2007


Prev   Next

Current Headlines

Papamanthou Awarded Grant to Study Security in Blockchains and Cryptocurrencies

Alumnus Ravi Tandon Receives 2018 Keysight Early Career Professor Award 

Dinesh Manocha Joins ECE as the Paul Chrisman Iribe Professor of Computer Science

Brendan Sheehy Receives Winston Family Honors Writing Award from UMD’s Honors College

Alum Mingyan Liu is PI for Multiscale Network Games of Collusion and Competition MURI

Symbiont Health, Co-founded by ECE Major Erich Meissner Wins Pitch Dingman Competition & Do Good Challenge

Three ECE Students Named Outstanding Graduate Assistants

Aneesh Raghavan wins Ann G. Wylie Dissertation Fellowship

News Resources

Return to Newsroom

Search News

Archived News

Events Resources

Events Calendar

Additional Resources

UM Newsdesk

Faculty Experts