Reliable and Efficient Application Scheduling on Edge, Fog and Cloud

Abstract

Cloud computing has emerged in the last decade as a popular distributed computing service offered by commercial providers. Public Clouds offer pay-as-you-go access to elastic resources that can be acquired and released on-demand. Among IaaS providers, on demand Virtual Machines(VMs) give access to compute resources, and are one of the frequently used services. While these fi xed-price VMs have absolute reliability and availability, spot-priced VM offer much higher discount while being preemptible, trading-off reliability. For users running large workloads with many tasks on the Cloud, such deep discounts will be valuable, and should motivate them to incorporate spot VMs. However, when users want the benefi ts of both reliability and cost reduction, they require scheduling strategies for managing their workloads when running on spot VMs. More broadly, when we consider resources beyond the Cloud and in the Fog and Edge layers of the network, such unreliability and transience become common due to commodity hardware, network variability and device mobility. Scheduling single tasks, bags of tasks or data flows on such dynamic Edge, Fog and Cloud resources with diverse capacities is challenging. We propose techniques for reliable and efficient application scheduling on these resources. First, we propose an automated scheduling engine to manage decisions related to Cloud VMs. Specifically, AutoBoT manages spot and fixedprice VM acquisition, bid price, and release, and task placement, checkpointing and migration for a Bag of Tasks(BoT), within a guaranteed completion time specifi ed by the user while minimizing the cost paid by them to the Cloud provider. The use of spot-priced VMs enhances the pro t relative to running exclusively on reliable VMs. We also propose various checkpointing strategies. The experimental results show that our scheduler gives 80% pro t and rare but bounded losses, compared to using only fixed-price VMs. Further, its 100% completion guarantee is 23 ð€€€ 42% better than using only spot-priced VMs which offer a similar pro t. Second, the heuristics proposed for AutoBoT are extended to transient Edge and Fog resources. We propose heuristics for distributed scheduling of tasks of DAGs on mobile Edge resources, Fog devices and Cloud VMs such that the overall DAG's deadline is met and total monetary cost of the resources billed is minimized.