Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system
As recording technology improves and becomes more affordable, many learning institutions are using lecture recording to make lessons more persistent and accessible. Statically mounted 4K cameras are now cheaper than PTZ cameras which makes them a desirable alternative for lecture recordings. Unfortu...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Computer Science
2024
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613251077931008 |
|---|---|
| access_status_str | Open Access |
| author | Khatieb, Mohamed Tanweer |
| author2 | Marais, Patrick
|
| author_browse | Khatieb, Mohamed Tanweer Marais, Patrick |
| author_facet | Marais, Patrick
Khatieb, Mohamed Tanweer |
| author_sort | Khatieb, Mohamed Tanweer |
| collection | Thesis |
| description | As recording technology improves and becomes more affordable, many learning institutions are using lecture recording to make lessons more persistent and accessible. Statically mounted 4K cameras are now cheaper than PTZ cameras which makes them a desirable alternative for lecture recordings. Unfortunately, 4K resolution videos are very large, posing a problem for storage and streaming - the file size for a 45 - 60 minute lecture video in 4K can exceed 2GB. Many students cannot afford the bandwidth required to stream such large files. Furthermore, since static 4K cameras do not move, they require a wide-angle view of the venue in order to capture as much of the front of the venue as possible. This view is much too zoomed out for viewers to see the details, such as writing on the boards and the presenter's facial expressions, captured by the 4K resolution. This dissertation investigates an approach to post-processing these 4K lecture videos to reduce the file size and emphasise lecture details such as lecture motion and board/screen usage. This is done using scene tracking data (generated via a third-party front-end) which a Virtual Cinematographer (VC) uses to make decisions on about which areas to crop from each 4K frame in the original video. The VC then positions and sizes the cropping windows in such a way that the resultant, cropped video resembles one recorded by a human camera operator. This is accomplished using cinematographic heuristics to inform its decision-making. The VC uses scene analysis algorithms to determine how the environment changes as time progresses in the video. By dividing the video into “chunks” (equivalent to “scenes” in traditional cinematography) based on context, the VC is able to maintain stable shots with consistent framing to avoid jittery and disorienting footage. These contextual chunks are determined by comparing the trajectory of the presenter with the manner in which the features on the board regions change over time. After the chunks are established, the VC creates transitions between them while avoiding any changes to the framing inside each chunk. The final output is a JSON file containing the cropping coordinates for each frame in the video for a third-party video cropping application to use when producing the final video. We performed a user evaluation of the VC to measure user satisfaction with the resulting output videos and how successful it was at following its heuristics. The VC succeeded in following the major heuristics such that viewers were satisfied with the output based on the framing of the presenter and the content on the boards, transition stability and smoothness of motion, and transition frequency with the VC only changing shots when necessary. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/40692 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:33:10.259Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2024 |
| publishDateRange | 2024 |
| publishDateSort | 2024 |
| publisher | Department of Computer Science |
| publisherStr | Department of Computer Science |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/40692 Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system Khatieb, Mohamed Tanweer Marais, Patrick Marquard, Stephen Marquard, Stephen Computer Science As recording technology improves and becomes more affordable, many learning institutions are using lecture recording to make lessons more persistent and accessible. Statically mounted 4K cameras are now cheaper than PTZ cameras which makes them a desirable alternative for lecture recordings. Unfortunately, 4K resolution videos are very large, posing a problem for storage and streaming - the file size for a 45 - 60 minute lecture video in 4K can exceed 2GB. Many students cannot afford the bandwidth required to stream such large files. Furthermore, since static 4K cameras do not move, they require a wide-angle view of the venue in order to capture as much of the front of the venue as possible. This view is much too zoomed out for viewers to see the details, such as writing on the boards and the presenter's facial expressions, captured by the 4K resolution. This dissertation investigates an approach to post-processing these 4K lecture videos to reduce the file size and emphasise lecture details such as lecture motion and board/screen usage. This is done using scene tracking data (generated via a third-party front-end) which a Virtual Cinematographer (VC) uses to make decisions on about which areas to crop from each 4K frame in the original video. The VC then positions and sizes the cropping windows in such a way that the resultant, cropped video resembles one recorded by a human camera operator. This is accomplished using cinematographic heuristics to inform its decision-making. The VC uses scene analysis algorithms to determine how the environment changes as time progresses in the video. By dividing the video into “chunks” (equivalent to “scenes” in traditional cinematography) based on context, the VC is able to maintain stable shots with consistent framing to avoid jittery and disorienting footage. These contextual chunks are determined by comparing the trajectory of the presenter with the manner in which the features on the board regions change over time. After the chunks are established, the VC creates transitions between them while avoiding any changes to the framing inside each chunk. The final output is a JSON file containing the cropping coordinates for each frame in the video for a third-party video cropping application to use when producing the final video. We performed a user evaluation of the VC to measure user satisfaction with the resulting output videos and how successful it was at following its heuristics. The VC succeeded in following the major heuristics such that viewers were satisfied with the output based on the framing of the presenter and the content on the boards, transition stability and smoothness of motion, and transition frequency with the VC only changing shots when necessary. 2024-11-08T08:54:58Z 2024-11-08T08:54:58Z 2023 2024-09-03T09:33:36Z Thesis / Dissertation Masters Masters http://hdl.handle.net/11427/40692 eng application/pdf Department of Computer Science Faculty of Science |
| spellingShingle | Computer Science Khatieb, Mohamed Tanweer Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| thesis_degree_str | Master's |
| title | Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| title_full | Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| title_fullStr | Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| title_full_unstemmed | Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| title_short | Investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post-processing system |
| title_sort | investigating the virtual directing strategies of a virtual cinematographer in an automatic lecture video post processing system |
| topic | Computer Science |
| url | http://hdl.handle.net/11427/40692 |
| work_keys_str_mv | AT khatiebmohamedtanweer investigatingthevirtualdirectingstrategiesofavirtualcinematographerinanautomaticlecturevideopostprocessingsystem |