Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Mixed-Criticality Scheduling Using Reinforcement Learning
Mixed-criticality (MC) scheduling is necessary for many safety-critical real-time embedded systems, as a failure of high-criticality jobs could lead to fatal accidents. With the emergence of software technologies in software-defined vehicles in the automotive and avionics industries, studying Mixed-...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
AUC Knowledge Fountain
2023
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613422259011584 |
|---|---|
| access_status_str | Open Access |
| author | ElSeadawy, Omar |
| author_browse | ElSeadawy, Omar |
| author_facet | ElSeadawy, Omar |
| author_sort | ElSeadawy, Omar |
| collection | Thesis |
| description | Mixed-criticality (MC) scheduling is necessary for many safety-critical real-time embedded systems, as a failure of high-criticality jobs could lead to fatal accidents. With the emergence of software technologies in software-defined vehicles in the automotive and avionics industries, studying Mixed-Critically (MC) systems is essential to their safety standards, similar to ISO26262. The real-time operation of MC systems makes it an inherently online problem, such that the scheduler is only aware of the jobs that are currently released at any point in time and has no knowledge of future jobs. Due to the overhead cost of preemption, this study focuses on enforcing non-preemption, which makes the problem NP-hard. The literature presents solutions for offline models that allow the scheduler to know about all jobs that are yet to be scheduled from time unit zero and also for systems that allow preemption. Researchers also simplify the modeling of the dynamic elements of the problem, e.g., varying-speed processors, by using simple assumptions that the processor's speed doesn’t recover from degradation, which simplifies the problem but is not very realistic. To the extent of our knowledge, we are the first to schedule dual-criticality systems upon non-preemptive, varying-speed processors online. With plenty of researchers approaching the schedulability of such systems with various objectives, our aim in this study is to shed light on the promising nature of emergent machine learning technologies, specifically Reinforcement Learning. We propose a somewhat unconventional approach, where we tackle the modeling complexities using deep reinforcement learning, particularly suitable for problems that generate a sequence of decisions in dynamic environments. Our customized Ape-X model is capable of successfully scheduling sets of jobs of size 50 with an average accuracy of 95% in comparison to other Reinforcement learning algorithms benchmarks conducted, e.g., Augmented Random Search, Proximal Policy Optimization, and Deep Q Networks. Sensitivity analysis shows that training the model with randomized parameters yields a stable performance that is relatively robust to some changes in the generated instances. As part of our future work, we also introduced a simple preemptive version of our system and showed its potential, which reached an average accuracy of 96%. We hope that our study and results motivate the scheduling community to explore the adoption of this effective approach as a promising potential for other dynamic scheduling problems. Thus, we also introduce our recommendations on modeling variants of the problem and discuss possible future extensions. |
| format | Thesis |
| id | oai:fount.aucegypt.edu:etds-3110 |
| institution | American University in Cairo (Egypt) |
| last_indexed | 2026-06-10T12:35:53.165Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from AUC Knowledge Fountain — bepress |
| publishDate | 2023 |
| publishDateRange | 2023 |
| publishDateSort | 2023 |
| publisher | AUC Knowledge Fountain |
| publisherStr | AUC Knowledge Fountain |
| record_format | dspace |
| source_str | AUC Knowledge Fountain — bepress |
| spelling | oai:fount.aucegypt.edu:etds-3110 Mixed-Criticality Scheduling Using Reinforcement Learning ElSeadawy, Omar Mixed-criticality (MC) scheduling is necessary for many safety-critical real-time embedded systems, as a failure of high-criticality jobs could lead to fatal accidents. With the emergence of software technologies in software-defined vehicles in the automotive and avionics industries, studying Mixed-Critically (MC) systems is essential to their safety standards, similar to ISO26262. The real-time operation of MC systems makes it an inherently online problem, such that the scheduler is only aware of the jobs that are currently released at any point in time and has no knowledge of future jobs. Due to the overhead cost of preemption, this study focuses on enforcing non-preemption, which makes the problem NP-hard. The literature presents solutions for offline models that allow the scheduler to know about all jobs that are yet to be scheduled from time unit zero and also for systems that allow preemption. Researchers also simplify the modeling of the dynamic elements of the problem, e.g., varying-speed processors, by using simple assumptions that the processor's speed doesn’t recover from degradation, which simplifies the problem but is not very realistic. To the extent of our knowledge, we are the first to schedule dual-criticality systems upon non-preemptive, varying-speed processors online. With plenty of researchers approaching the schedulability of such systems with various objectives, our aim in this study is to shed light on the promising nature of emergent machine learning technologies, specifically Reinforcement Learning. We propose a somewhat unconventional approach, where we tackle the modeling complexities using deep reinforcement learning, particularly suitable for problems that generate a sequence of decisions in dynamic environments. Our customized Ape-X model is capable of successfully scheduling sets of jobs of size 50 with an average accuracy of 95% in comparison to other Reinforcement learning algorithms benchmarks conducted, e.g., Augmented Random Search, Proximal Policy Optimization, and Deep Q Networks. Sensitivity analysis shows that training the model with randomized parameters yields a stable performance that is relatively robust to some changes in the generated instances. As part of our future work, we also introduced a simple preemptive version of our system and showed its potential, which reached an average accuracy of 96%. We hope that our study and results motivate the scheduling community to explore the adoption of this effective approach as a promising potential for other dynamic scheduling problems. Thus, we also introduce our recommendations on modeling variants of the problem and discuss possible future extensions. 2023-06-01T07:00:00Z thesis application/pdf https://fount.aucegypt.edu/etds/2076 https://fount.aucegypt.edu/context/etds/article/3110/viewcontent/Thesis___Mixed_Criticality_Scheduling_AUC_Fount_Submission.pdf Theses and Dissertations AUC Knowledge Fountain Mixed criticality systems Varying speed processors Deep reinforcement learning Online dynamic scheduling Non-preemptive scheduling Computer Engineering |
| spellingShingle | Mixed criticality systems Varying speed processors Deep reinforcement learning Online dynamic scheduling Non-preemptive scheduling Computer Engineering ElSeadawy, Omar Mixed-Criticality Scheduling Using Reinforcement Learning |
| title | Mixed-Criticality Scheduling Using Reinforcement Learning |
| title_full | Mixed-Criticality Scheduling Using Reinforcement Learning |
| title_fullStr | Mixed-Criticality Scheduling Using Reinforcement Learning |
| title_full_unstemmed | Mixed-Criticality Scheduling Using Reinforcement Learning |
| title_short | Mixed-Criticality Scheduling Using Reinforcement Learning |
| title_sort | mixed criticality scheduling using reinforcement learning |
| topic | Mixed criticality systems Varying speed processors Deep reinforcement learning Online dynamic scheduling Non-preemptive scheduling Computer Engineering |
| url | https://fount.aucegypt.edu/etds/2076 https://fount.aucegypt.edu/context/etds/article/3110/viewcontent/Thesis___Mixed_Criticality_Scheduling_AUC_Fount_Submission.pdf |
| work_keys_str_mv | AT elseadawyomar mixedcriticalityschedulingusingreinforcementlearning |