Professional Documents
Culture Documents
TELECOMMUNICATION ENGINEERING DEPT. UNIVERSITY OF ENGINEERING AND TECHNOLOGY, TAXILA July, 2013
[1]
[2]
ABSTRACT
Data is driving todays ICT industry and research. The rate at which data are being generated is immense, and its soaring everyday. The database storage systems in most organizations and research institutions no longer totally capable of storing ever increasing data. The phenomenon that is rising as a solution to this problem is cloud based storage. The industry and leading research organizations are increasingly adopting cloud storage. There are several cloud storage is available commercially, these are very good as well. But these services are proprietary and in the public domain. An institution such as HEC cant be trusted over those services completely. This motivates us to develop an indigenous and private cloud storage system for HEC and its network of universities PERN, based on the architectures of public services.
[3]
UNDERTAKING
We certify that research work titled Design and Development of Cloud based storage for academia is our own work. The work has not, in whole or in part, been presented elsewhere for assessment. Where material has been used from other sources it has been properly acknowledged/ referred.
(09-TE-01) (09-TE-25)
[4]
ACKNOWLEDGEMENTS
Firstly, thanks Allah Almighty with His blessing for our Design and Development of Cloud based storage for academia using PERN resources project. We would like to express our sincere appreciation to our supervisor, Dr. Adeel Akram for his total support, guidance, advice and assistance throughout the process of this final year project. We are very grateful to Mr. Abdul Fayyaz Chatta, Director IT, and PERN for his time and guidance. He. The suggestions provided by him really proved helpful. We would also like to take this opportunity to thank our beloved parents and siblings for always mentally and financially supporting us while doing this project. Our fellow course mates should also be recognized for their support. Our sincere appreciation is also extended to all our friends, teachers and others who have provided assistance at various occasions. Their views and tips are useful indeed
[5]
Table of Contents
ABSTRACT ................................................................................................................... 3 UNDERTAKING ............................................................................................................. 4 ACKNOWLEDGEMENTS ................................................................................................ 5 Table of Figures ........................................................................................................... 8 abbreviations .............................................................................................................. 9 Chapter 1: Introduction ............................................................................................. 10 1.1 1.2 1.3 Problem Statement ...................................................................................... 10 Objectives .................................................................................................... 10 Background ................................................................................................. 10 Characteristics of Cloud......................................................................... 11 Service Models .................................................................................. 12 Deployment Models .............................................................................. 13
1.3.1 1.3.2 1.3.3 1.4 1.5 1.6 1.7 2.1 2.2 2.3
Our Model: .................................................................................................. 14 Deliverables ................................................................................................. 14 Tools Used ................................................................................................... 14 Organization of Thesis ................................................................................. 15 An Autonomic Frame of Mind ...................................................................... 17 Features of Cloud ........................................................................................ 17 Characteristics of an autonomic service architecture ................................... 20 Architecture style .................................................................................. 20 External user and access control management ..................................... 21 Interaction container............................................................................. 21 Externalized policy management/policy Engine .................................... 22 Utility Computing .................................................................................. 23
Chapter 3: Methodology ............................................................................................ 24 PLAN ............................................................................................................ 24 Cloud Infrastructure Software: .............................................................. 24 Cloud management server/cloud controller: ......................................... 24 [6] 3.1.1 3.1.2
3.2 3.3
Client End Application.................................................................................. 25 ATL: ....................................................................................................... 25 Hypervisor ............................................................................................ 26 Hypervisor Server ................................................................................. 26 Management software .......................................................................... 27 Web 2.0 .................................................................................................... 27 REST: ..................................................................................................... 28 Server Side Implementation ........................................................................ 26
3.4.2.1 RESTful API .......................................................................................... 29 Flow of Design ............................................................................................. 29 Application Development ...................................................................... 29 Application Deployment........................................................................ 30 3.5.1 3.5.2
[7]
TABLE OF FIGURES
Figure 1-Service Models and Apps ............................................................................ 12 Figure 2-Private Cloud .............................................................................................. 13 Figure 3-Public Cloud ................................................................................................ 14 Figure 4-Cloud Drivers .............................................................................................. 18 Figure 5-Cloud Controler........................................................................................... 25 Figure 6-Hypervisor Type-1 ....................................................................................... 26 Figure 7-VSpehere Implementation Model ............................................................... 27 Figure 8- Application Development Flow .................................................................. 29 Figure 9- Application Deployment Flow .................................................................... 30
[8]
ABBREVIATIONS
IaaS: Infrastructure As A Service PaaS: Platform As A Service SaaS: Software As A Service VM: Virtual Machine PERN: Pakistan Education Research network SQL: Sequential Language HTTP: Hyper Text Transfer Protocol REST: Representational State Transfer API: Application Programming Interface ATL: Active Template Library NSE: NameSpace Shell Extension LAMP: Linux Apache MySQL PHP JSON: JavaScript Object Notation SDK: Software Development Kit
[9]
CHAPTER 1: INTRODUCTION
Cloud Computing is the top trend among consumers, businesses and in the technology industry. People are adopting it as a way of life. The days are gone when the cloud is termed as a large data warehouse, now its a reality and people are moving to the cloud increasingly. This serves as an opportunity for the industry as well as businesses alike, seeing that many cloud services provided in the public market many of which are well known players in the industry like Google and Microsoft. The publicly provided cloud services serves as cost effective in the short term but have some serious issues when it comes to security. Connectivity is also another issue because you are connected through the internet. The sensitive and high priority data like academic research cant be trusted in public domain. Our aim is to develop a cloud that provide the public cloud like services like SkyDrive etc. but resides in the on-premise network. This offers high security with high speed connectivity as it will be using PERN network which is in Gigabit range. It is also integrated with the users windows. [1]
1.1
Problem Statement
People are now mobile than ever before and require their I.T. applications and data to be ubiquitous and device independent. Cloud is the answer to this, but if you have confidential data that cant be uploaded in public domain and over the third party data centres, Private Cloud on the organizations resources is the solution.
1.2
Objectives
1. Design Private Cloud storage. 2. Integrate it with the UET, Taxilas servers. 3. Create users roaming profile so that they can access data from anywhere and from any device. 4. System is fully scalable if demand increases.
1.3
Background
Cloud computing is an evolving paradigm. The NIST is the international body for the standards in technology. The NIST definition of cloud characterizes important aspects of cloud computing and is intended to serve as a means for broad comparisons of [10]
cloud services and deployment strategies, and to provide a baseline for discussion from what is cloud computing to how to best use cloud computing. Cloud computing is a model for enabling ubiquitous, convenient, on -demand network access to a shared pool of configurable computing resources ee.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. This cloud model is composed of five essential characteristics, three service models, and four deployment models. [2]
[12]
Community cloud.
The cloud infrastructure is provisioned for exclusive use by a specific community of consumers from organizations that have shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be owned, managed, and operated by one or more of the organizations in the community, a third party, or some combination of them, and it may exist on or off premises.
Public cloud.
The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider.
Hybrid cloud.
The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).
[13]
1.4
Our Model:
We are developing Private cloud as a deployment model and providing Infrastructure as a Service (IaaS). Our client, HEC, is a research organization, which has an infrastructure need at most.
1.5
Deliverables
Cloud based Storage, incorporating total standard functionality and usability, based on commercially available applications, which serve as private ubiquitous storage for PERN & the HEC network in Pakistan.
1.6
Tools Used
Microsoft Visual Studio JavaScript SDK VMware ESXi Hypervisor VMware Vsphere [14]
1.7
Organization of Thesis
In First Chapter the introduction of the project is given and background is discussed. Second chapter presents the literature review about the Cloud Technology, its working principle, case studies, and windows explorer applications for usability server implementation. Third chapter relates to the methodology that will be adopted for the completion of the project. In the fourth chapter the programming and simulations of project in Microsoft Visual studio and Server implementation are given.
[15]
[16]
2.1
The architectures that we deploy in data centers today should be able to run in a cloud, simply moving them into a cloud stops well short of what one might hope that Cloud Computing will come to mean. [6]In fact, tackling global-scaled collaboration and trading partner network problems in government, military, scientific, and business contexts will require more than what current architectures can readily support. For example: It will be necessary to rapidly set up a temporary collaboration network enabling network members to securely interact online, where interaction could imply interoperability with back office systems as well as human oriented exchanges all in a matter of hours. Business interactions have the potential to become more complex than personal transactions. The way that users and access control are managed in typical applications today is no longer flexible enough to express roles and responsibilities that people will play in next-generation business interactions.
2.2
Features of Cloud
The above mentioned considerations suggest that cloud will have to have at least the following characteristics: Clouds should be uniquely identifiable so that they can be individually managed even when combined with other clouds. This will be necessary to distinguish and harmonize cloud business and infrastructure policies in force. A cloud should be dynamically configurable: configuration should be automatable in varying and unpredictable, possibly even event -driven, conditions. Systems management technologies for clouds must integrate constraints on business with constraints on infrastructure to make them manageable in aggregate. A cloud should be able to dynamically provision itself and optimize its own construction and resource consumption over time. A cloud must be able to recover from routine and extraordinary events that might cause some or all of its parts to malfunction.
[17]
A cloud must be aware of the contexts in which it is used so that cloud contents can behave accordingly.
These coarse-grained characteristics, sometimes described as autonomic computing, can be represented in the form of finer -grained architecture drivers that are useful in characterizing steps toward an autonomic computing architecture. [7] Cloud Computing offerings that are available today share many of the same drivers that we have organized into Systems and Application Management Drivers in the figure below. Numbered circles in the graphic above denote drivers that are listed below:
0. Architecture state: no systems management 1. Systems and resources must be identifiable 2. System and resources must be manageable 3. Policy-driven secured access to the system and managed resources must be provided [18]
4. System must reallocate managed resources on failures as a function of policy 5. System must reallocate managed resources on various system -level conditions by policy 6. System must be managed lights-out in a single data center context 7. Systems management capability must scale across clouds of the same type 8. Systems management capability must scale across clouds of different types; these clouds must be managed uniformly while maintaining separate cloud identities 9. System must reallocate managed resources on various system-level conditions as a function of policy to accommodate real -time and business-oriented usage patterns 10. Systems management policies are harmonized across cloud boundaries 11. It must be possible to integrate management policies of different clouds 12. Monolithic applications and traditional application integrations exist/are sufficient 13. Application platform must be service oriented [8] 14. Applications are replaced with business services 15. Business services have secured access 16. An Interaction Container1 must be used as application container in a singletenant environment 17. Policies must be consolidated and managed using a single (possibly federated) policy engine 18. System must reallocate managed business services on various business level conditions by policy to accommodate real-time/batch usage patterns 19. An Interaction Container must be used as application container in a multitenant environment 20. Business service and systems management policies are integrated 21. Architecture state: positioned as an autonomic architecture platform for virtual organization-oriented application systems 22. Architecture state: additional structural and business constraints positioning architecture platform as a service grid
[19]
The graph depicts two paths toward autonomic computing that ultimately converge at an architecture point that could support business ecosystems and emergent and fluid virtual organizations: The first path, Systems Management Drivers, begins with no systems management, and ends with a systems management capability that is policy driven, and that enables automated systems management in a cloud and harmonization of business and infrastructure policies within and across cloud boundaries in both single- and multi-tenant modes. The drivers for systems management are grouped to illustrate needs common to basic systems management (Systems Management Capabilities), and needs that go beyond the basic capabilities (Utility Computing Management and Outside-In Architectureiii Capabilities). [9] The second path, Applications Management Drivers, begins with common monolithic corporate applications. It ends with these applications having been replaced with a service-oriented ones, where policy has been externalized so that business policies can be harmonized with utility management policies, where it is possible to implement end-to-end service level agreements and enforce conformance to business and regulatory constraints, and where the use of business functional and infrastructural components can be metered and elastically load balanced. At this endpoint, business services and infrastructure can be organized into a cloud and used in both single- and multitenant modes. [10]
2.3
As cloud computing solutions and products are implemented, we believe it critical especially to those being driven by their business needs up the Systems and Applications Management Drivers curves [11] to carefully consider their need for support of the architecture characteristics elaborated ass below:
application-oriented; factors out policy as a first class architecture component that can be used to govern transparent performance of service -related tasks; and emphasizes the ability to adapt performance to user/business needs without having to consider the intricacies of architecture workings. The counter style, what we call inside-out, is inherently bottom-up and takes much more of an infrastructural point of view as a starting point, building up to a business functional layer. Application platforms constructed using client server, objectoriented, and 2/3/n-tier architecture styles are those to which we apply the generalization inside-out because they form the basis of enterprise application architectures today, and because architectures of these types have limitations that require transformation to scale in a massive way vis --vis outside-in platforms. Implementation of an outside -in architecture results in better architecture layering and factoring, and interfaces that become more business than data oriented. [13]
transparently, in many cases to business logic implemented as classes of behavior as long as it was implemented to conform to special (e.g., bean) interfaces, freeing developers to focus on implementing business functionality and not infrastructure resulting in a standardized use of infrastructure services. Containers are hosted in application servers. The term Interaction Container is used as an analog to J2EE/Java EE application container. The Interaction Container is hosted in an Interaction Server, statically and dynamically configured to provide infrastructure and policy adjudication services that are specific to a business users environment, integrated with systems management capabilities, and used to manage one-to-many Interactions and their life cycles. An Interaction Container essentially holds an execution context in which role players people or systems participating in an Interaction and conforming to specific roles (interfaces) interact to perform their parts in a business orchestration and manage exceptions and/or faults should they occur in the process. An Interaction Container can be considered to be organized based (i.e., it can be used to manage many Interactions between a set of participants/role players over time), or outcome-based (in which only one Interaction would be performed). These two usage scenarios reflect the need to manage Interactions in a dynamic user community, where role players could change over time, and the need to manage an Interaction as a single possible long-running business transaction.
[22]
[23]
CHAPTER 3: METHODOLOGY
In this chapter we will give clarification about the method that is used in our project. It includes the entire project plan, explanation of the design flow and also design architecture.
3.1
PLAN
This section includes the whole project plan, its basic architecture, its working methodology and architectures of different parts like sensor node, gateway node and control room equipment. We are basically using the Self-Service IT portal for the cloud. We are dealing with following:
[24]
The architecture shown in Figure can be expanded to include multiple clusters that add both capacities to the solution as well as redundancy that can be used to increase the overall availability of the infrastructure. The foremost task of this project is to create Server Templates. Server Templates (a configuration blueprint for a server running on a cloud) and made these templates accessible to the business units end users. The end users logged in to the Dashboard of the cloud interface and selected one of these preconfigured Server Templates. In a matter of minutes, they provisioned a fully configured and available server for their test and development purposes. Because each of these server instances was independent, an end user could perform whatever actions he needed without being concerned about modifying the environment of another end user. When development and testing was completed, the end user terminated the server instance, which reduced operational costs and freed up resources that could be repurposed for another user.
3.2
The application is JavaScript and html based. It accesses the SkyDrive architecture of Microsoft through its API. The application acts the the interface between the user computer and the database cloud storage. Application is integrated with the users windows profile via shell namespace extension. This is done through windows ATL libraries. 3.2.1 ATL: The Active Template Library (ATL) is a set of template -based C++ classes developed by Microsoft, intended to simplify the programming of Component Object Model (COM) objects. The COM support in Microsoft Visual C++ allows developers to create a variety of COM objects, OLE Automation servers, and ActiveX controls. ATL includes an object wizard that sets up primary structure of the objects very quickly with a minimum of hand coding. On the COM client side ATL provides smart pointers that deal with COM reference counting. [25]
3.3
3.3.1 Hypervisor
In computing, a hypervisor or virtual machine monitor (VMM) is a piece of computer software, firmware or hardware that creates and runs virtual machines. A computer on which a hypervisor is running one or more virtual machines is defined as a host machine. Each virtual machine is called a guest machine. The hypervisor presents the guest operating systems with a virtual operating platform and manages the execution of the guest operating systems. Multiple instances of a variety of operating systems may share the virtualized hardware resources. [19]
Figure 6-Hypervisor Type-1
[26]
3.4
Client-Server Interfacing
The cloud based application model we have followed is the web 2.0.
[27]
The key features of Web 2.0 include: 1. 2. 3. 4. 5. 6. 7. Folksonomy; free classification of information A rich user experience A user as a contributor Long tail User participation Basic trust Dispersion
The client-side (web browser) technologies used in Web 2.0 development include Ajax and JavaScript framework. The data fetched by an Ajax request is typically formatted in XML or JSON (JavaScript Object Notation) format; two widely used structured data formats. Since both of these formats are natively understood by JavaScript, a programmer can easily use them to transmit structured data in their web application. For example, Google Docs uses this technique to create a Web based word processor. Web 2.0 can be described in three parts: Rich Internet application (RIA) defines the experience brought from desktop to browser whether it is from a graphical point of view or usability point of view. Some buzzwords related to RIA are Ajax and Flash. Web-oriented architecture (WOA) is a key piece in Web 2.0, which defines how Web 2.0 applications expose their functionality so that other applications can leverage and integrate the functionality providing a set of much richer applications. Examples are fed, RSS, Web Services, mash-ups. Social Web defines how Web 2.0 tends to interact much more with the end user and make the end-user an integral part. Using web 2.0 technology enables us to incorporate REST architecture. In fact, this client/server relationship is a prerequisite of a set of principles called REST (or Representational State Transfer).
3.4.2 REST:
Representational state transfer (REST) is a style of software architecture for distributed systems such as the World Wide Web. REST has emerged as a predominant web API design model. Key goals of REST include: Scalability of component interactions Generality of interfaces Independent deployment of components [28]
3.5
Flow of Design
According to our requisite, we divide the design into two parts: 1) Application Development 2) Application Deployment
[29]
[30]
CONCLUSION
The project is based on the development of a private cloud ensuring SkyDrive like services. It allocates user's configurable computing resources that are ubiquitous and scalable. This is done through a client -side application integrated with windows explorer through roaming user profiles. The application is a service independent due to its web 2.0 and REST architecture. Its RESTful API can be linked to various cloud services.
[31]
FUTURE WORK
To deploy the prototype of a cloud storage system at national level, on PERN architecture. This will create a huge breakthrough in education research in Pakistan.
[32]
REFRENCES
[1] A. F. R. G. Michael Armbrust, "Above the Clouds: A Berkeley View of Cloud," Electrical Engineering and Computer Sciences, 2009. [2] National Institute of Standards and Technology, "NIST Cloud Computing Reference Architecture," NIST, 2011. [3] Arista, "Cloud Storage Whitepaper," www.aristanetworks.com. [4] Ericsson, "Operator opportunities in cloud service delivery," 2012. [5] Delloite, "Cloud computing: A collection of working papers," 2009. [6] tango telecom, "ENABLING EXTREME COMPETITION," 2012. [7] P. A. A. P. C. Bengt Ahlgren, "Content, Connectivity, and Cloud: Ingredients for the Network of the Future," p. 22. [8] M. K. Vishal Jain, "Information Retrieval through Multi -Agent System with Data Mining in Cloud Computing," IJCTA, p. 5, 2012. [9] E. D. Gbor Fodor, "Design Aspects of Network Assisted Design Aspects of Network Assisted," IEEE communication magazine, p. 8, 2012 . [10] Verivue, "Designing a Cloud Storage System," www.verivue.com/object -store. [11] K. Ferguson-Boucher, "Cloud Computing: A Records and Information Management Perspective," IEEE security and privacy, 2011. [12] IDC, "Mobile Virtualization: Accelerating Innovation in Next -Generation Services," 2012. [13] M. Alendal, "The premium cloud: How Operators can maximize profit". [14] Ericsson, "Enabling the network-embedded cloud," 2012. [15] A. K.-H. Ilango Sriram, "Research Agenda in Cloud Technologies". [16] Ericsson Review, "Transport networks in the Cloud Age," 2012. [17] Ericsson, "Deploying telecom-grade Cloud," 2012.
[33]
[18] R. Brian Adler, "Designing Private and Hybrid Clouds," 2012. [19] R. Vanover, "Virtualization review," [Online]. Available: http://virtualizationreview.com/blogs/everyday -virtualization/2009/06/type -1and-type-2-hypervisors-explained.aspx. [Accessed 04 01 2013]. [20] VMware, "VMware Infrastructure 3," 2006. [21] VMware, "VMware vSphere 5 Competitive Reviewers Guide," [Online]. Available: http://www.vmware.com/files/pdf/VMware-vSphere-CompetitiveReviewers-guide-WP-EN.pdf. [Accessed 02 07 2013]. [22] Microsoft, "ATL refrence," [Online]. Available: http://msdn.microsoft.com/en us/library/t9adwcde%28v=vs.80%29.aspx. [Accessed 2 7 2013]. [23] Microsoft, "Skydrive API," [Online]. Available: http://msdn.microsoft.com/en US/library/live/hh826521. [Accessed 02 07 2013].
[34]