Nowadays, universities are looking to solve the timetabling problem for scheduling the data in line with complex constraints on the existing resources in the environments, such as the subjects, lecturers, students, and classroom specifications. This thesis aims to develop a framework for optimizing resource allocation to university timetabling problems for generating schedules. Systematic literature investigations are conducted into university course timetabling, and deep reinforcement learning to the timetabling problems to help for decision making to the proposed techniques and algorithms. This study focuses on the lecturer’s schedules based on soft and hard constraints with existing resources using reinforcement learning with a multiagent system. This thesis conducted the experimental methodology using a dataset from Umm Al-Qura University, which had a number of specific soft and hard constraints based on its environment analyses. These constraints are used in Q-learning, having a Q-table to find the best reward through a sequential iteration on the states to get the actions through a number of agents to reach the better optimization to the course timetabling for lecturers’ scheduling.