Archive for April, 2014

Increase MySQL connections at run time without restarting the server – set global max_connections = 500

mysql>  select @@max_connections;
+——————-+
| @@max_connections |
+——————-+
|               151 |
+——————-+
1 row in set (0.00 sec)

mysql>  set global max_connections = 500;
Query OK, 0 rows affected (0.03 sec)

mysql>  select @@max_connections;
+——————-+
| @@max_connections |
+——————-+
|               500 |
+——————-+
1 row in set (0.00 sec)

View logs with colour
tail -f repository/logs/wso2carbon.log | awk ‘
{print “33[32m” $0 “33[39m”}
/ERROR/ {print “33[31m” $0 “33[39m”}
/FATAL/ {print “33[31m” $0 “33[39m”}
/WARN/ {print “33[33m” $0 “33[39m”}

 

Advertisements

Let’s assume you are running a SOAP service at the URL   http://localhost:8081/axis2/services/SimpleStockQuoteService?wsdl

SOAP UI  is a very good tool to access web services.

SoapUI 4.6.1 _095

However you have to install SOAP UI  or there may be situations where you don’t have access to tools, for instance when you have longed in to an server by SSH. Then you have access only to command line. Here come the curl command handy.

Let’s see how you can access the above web service using curl command.

curl  –header “Content-Type: text/xml;charset=UTF-8” –data @request.xml http://localhost:8081/axis2/services/SimpleStockQuoteService

(Please note that there are two (not one) dashes before parameters.)

request.xml files contains the request payload. Below is the payload for the SimpleStockQuoteService service.

<ser:getQuote xmlns:ser=”http://services.samples”&gt;
<!–Optional:–>
<ser:request>
<!–Optional:–>
<xsd:symbol>IBM</xsd:symbol>
</ser:request>
</ser:getQuote>

Use case 

Adding persistent volume is very much similar to adding a virtual hard drive to your instance. In Amazon EC2 it is adding a EBS storage device to the instance. Please refer to [1] for more details. Persistence volume capability comes handy when you want to store your content in a separate place and make the data available even after the instance is terminated or deleted. MySQL, MongoDB are examples where you may need this capability.

Below are the steps you need to perform to enable persistent volume mapping capability.

 Cartridge Definition

You need to add a similar configuration to the cartridge definition. Please note that the persistent is an optional configuration which user specify if he wants additional volumes to store his content.
 “persistence”: {
        “isRequired”: true,
        “volume” : [
            {
            “device”: “/dev/sdc”,
            “mappingPath”: “/home/ubuntu/sdc”,
            “size”: “10”,
            “removeOnTermination”: “false”
            }
     ]
     }
Here I say I want an extra volumes to store my content. Linux device of the first volume should be /dev/sdc which will be mounted to the directory /home/ubuntu/sdc. The capacity of the volume should be 10GB. removeOnTermination parameter specify what should be done to the volume created after the termination of the instance. if removeOnTermination=false the volume and its data will exist even after the volume is terminated so data is not deleted.
Subscribe with persistent volumes
Just by adding to the cartridge definition does not provide volumes for the instances. We have to specify we require volumes at the time of subscribing.

Subscribing via CLI

You have to add below highlighted parameters when subscribing by Stratos CLI.
subscribe-cartridge <other-parameters> -pv <PERSISTENCE-VOLUME> -v <VOLUME-SIZE> -t <REMOVE-ON-TERMINATION>
PERSISTENCE-VOLUME>= true/false.  true if you need the feature
VOLUME-SIZE = size (in Giga Bytes) of the storage needed
REMOVE-ON-TERMINATION = true/false whether to delete the volume when the instance is terminated.
Subscribing via Stratos UI
               If you are subscribing with the Stratos Ui, there is tick as “Require Persistent Storage mappings” on the “subscribe to cartridge” page. Please tick it and provide required details.
Subscribe with Persistent mapping

Subscribe with Persistent mapping

Multiple persistent volumes

Stratos provides the capability to specify multiple persistent volumes in case you require more that one additional volume.
 “persistent”: {
        “isRequired”: true,
        “volume” : [
            {
            “device”: “/dev/sdc”,
            “mappingPath”: “/home/ubuntu/sdc”,
            “size”: “10”,
            “removeOnTermination”: “false”
            },
            {
            “device”: “/dev/sdf”,
            “mappingPath”: “/home/ubuntu/sdf”,
            “size”: “20”,
            “removeOnTermination”: “false”
            }
     ]
     }
Note that don’t specify the same mappingPath to multiple volumes. Then it will be mapped to the last volume which can not be predicted, other volumes will not be mapped to a directory.

What happens behind the scene.

If you subscribe with persistent mappings, Stratos will,
1 Create the volumes.
2 Format and create a writable file system (ext3)
3 Mount volumes to the directory specified as mappingPath
Please note that the formatting and creating a file system happens at the first time only. Next time when the volume is attaching to another instance (after first instance is terminated), it is no longer needed to create a file system since file system is already there.

 How to verify if the volumes are created.

Log in to the instance created by ssh and execute the command “df -h”. You will see a output similar to below.
df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc        10G  7.1G   97G  3% /home/ubuntu/sdc
/dev/sda1       4000G  40G   320G  10% /home
Limitations.
In most IAASes such as EC2 one volume can be attached to only one instance. So if your deployment policy has maxInstance count more than 1, you might encounter problems when Stratos tries to attach the the volume to the second instance.
This has tested only in EC2, need to be tested in other IAASes as well.

Improvements

Currently size parameter is mandatory at the time of subscription. But in ideal case the size specified in cartridge definition should be considered as the default if the size is not given at the time of subscription.

Apache Stratos is a polyglot PaaS framework, providing developers a cloud-based environment for developing, testing, and running scalable applications. You can plug any service such as PHP, Tomcat, Ruby, MySQL, etc. as a cartridge at runtime. This tutorial will give you a step-by-step approach on how to create a PHP cartridge. We use OpenStack as the underlying Infrastructure as a Service (IaaS). The steps are similar in other IaaSs, such as Amazon EC2.

Introduction

Before creating the cartridge, puppet master has to be configured by using the following link; Apache Stratos Puppet Master setup guide

Live terminal recording of configuring a puppet master can be found through the link; live terminal recording of setting up

Let’s assume, the puppet master was configured with the values,

PUPPETMASTER-DOMAIN= puppet.udara.com

PUPPET-MASTER-IP=192.168.18.13

Creating the cartridge to Stratos

  1. Create an ubuntu VM in Openstack
  • Select the image with the OS and click Launch

Stratos_1

  • This will lead to a popup window called Launch instance where the Flavor has to be selected

Stratos_2

  • Choose a key pair

Stratos_3

2 .Log in to the machine created via SSH

ssh -i udara.pem ubuntu@192.168.18.130

 

3. Install unzip

sudo apt-get install unzip

This step is carried out to extract the Stratos Cartridge Agent before installing the Cartridge Agent

 

4. Create the configuration script in the path; /root/bin/config.sh with the content copied from this configuration file.

vim /root/bin/config.sh

Copy the content.

Make the script executable

chmod +x /root/bin/config.sh

This script installs the service you want (PHP in this case) and install and start the Puppet agent. Then puppet agent connects to puppet master and installs the other required software such as Apache2, PHP, Java and Stratos cartridge agent, etc.

 

5. Create the init script in /root/bin/init.sh with the content at this init script.

Copy and paste the content.

Make the script executable

chmod +x /root/bin/init.s

 

6. Create the puppet-install script in /root/bin/puppetinstall/puppetinstall with the content at this script.

vim /root/bin/puppetinstall/puppetinstall

Copy and paste the content

Make the script executable

chmod +x /root/bin/puppetinstall/puppetinstall

 

7. Create the configuration file /root/bin/stratos_sendinfo.rb with the content of stratos_sendinfo.rb

8. Now execute the config.sh

cd /root/bin

./config.sh

 

For the questions prompt in this step should be answered with the following values;

Since we are going to make a PHP cartridge

service-name=php

As we set up the puppet master in the machine which has the IP address of192.168.18.134

puppet master IP=192.168.18.134

As we setup the puppet master with the hostname

puppet master hostname = puppet.udara.com

root@udara-cartridge:~/bin# ./config.sh

This script will install and configure puppet agent, do you want to continue [y/n]y

–2014-04-01 04:04:52– http://169.254.169.254/latest/user-data

Connecting to 169.254.169.254:80… connected.

HTTP request sent, awaiting response… 404 Not Found

2014-04-01 04:04:52 ERROR 404: Not Found.

Please provide stratos service-name:php

Please provide puppet master IP:192.168.18.134

Please provide puppet master hostname [puppet.stratos.org]:puppet.udara.com

      This process may take some time. You may have to wait, till the process complete.

A log similar to below will be seen, once the above process is completed.

Info: Creating state file /var/lib/puppet/state/state.yaml

Notice: Finished catalog run in 2367.76 seconds.

 

9.Create the Snapshot

Now the machine is ready to become a cartridge. Required software is installed and configurations have been completed.

Stratos_4

Click on Create Snapshot button

 

Stratos_5

 

Save snapshot with a preferred name

Stratos_6

Snapshot is waiting to be created

 

Stratos_7

When the snapshot creation is completed, click on the snapshot to find details about it. The most important information is the ID of the image (snapshot).

When a PHP runtime is required, the user has to subscribe to a PHP cartridge in Stratos. When a PHP cartridge is subscribed, behind the scene, Stratos creates an instance from the image ID retrieved earlier. Since PHP is already installed and configured at the snapshot creating stage, PHP runtime will be available when Stratos spawn an instance from that snapshot.

 

Deploy the cartridge JSON in Stratos

Below is the cartridge JSON which is need to be deployed in Stratos. It is notable that the image ID is same as the snapshot ID, created above.

{
     "type": "php",
     "provider": "apache",
     "host": "udara.com",
     "displayName": "PHP",
     "description": "PHP Cartridge",
     "version": "5.0",
     "multiTenant": "false",
     "portMapping": [
        {
           "protocol": "http",
           "port": "80",
           "proxyPort": "8280"
        },
        {
           "protocol": "https",
           "port": "443",
           "proxyPort": "8243"
        }
      ],
     "deployment": {
       },
      "iaasProvider": [
    {
         "type": "openstack",
         "imageId": "RegionOne/ac859fc9-d0dc-4f4fb084-bada02ad5129",
         "maxInstanceLimit": "5",
         "property": [
       {
            "name": "instanceType",
            "value": "RegionOne/4"
           },
           {
            "name": "keyPair",
            "value": "stratos-s4"
           }
      ]
        }
     ]
}
Please refer to the cartridge deployment guide of Stratosfor more information.

timthumb

Earlier when people were thirsty, they had no other option other than digging a well. However with the population growth and urbanization, it was impractical to build wells to address each family’s need. Instead people created tanks where a huge volume of water could be stored. The stored water is supplied to the citizens of many towns and villages. This method has many visible advantages over the traditional way. All families don’t have to put an effort in digging wells for themselves, no need of time to maintain wells, and no dedicated and suitable place is required. An established and dedicated organization such as Water board does everything necessary to take water from a river, pump to a tank, purify and deliver to the end consumers. Consumers only pay for the volume of the water they consume which covers the overhead costs of the producer as well.

cloud-service

The above scenario is an analogy to the base concept of cloud computing. Earlier, when computing facilities or computer systems are required by an organization or an individual, they had to buy computers and the network infrastructure. In addition they had to buy licenses for necessary software and install them, allocate dedicated space and employ necessary personnel to maintain the system.  This requires a considerable initial setup cost and recurring maintenance cost. Scaling up and down based on market and business needs is not easy as it involves direct costs as well as management overheads associated with it. When the power of the internet was practically proven, the concept of cloud computing was emerged. As the water is received via pipes, computing power is received via internet. Thus the cloud computing can be explained as “computing via internet”.

FLAVORS OF CLOUD COMPUTING

Cloud computing comes in different flavors. Following will be discussed in this article.

  • IAAS – Infrastructure As A Service
  • SAAS – Software As A Service
  • PAAS – Platform As A Service

IAAS – Infrastructure as a Service

Dedicated or shared computing infrastructure is provided with the basic features such as the OS and other domain specific features installed. Users see these services as virtual machines with comprehensive administrative powers or hosting service with limited administration capabilities. Amazon EC2 is the most popular IAAS provider providing virtual computing power in the cloud. There are thousands of web hosting service providers such as godaddy.comhostgater.comixwebhosting.com.

For instance let’s consider Amazon EC2. After creating an account in Amazon EC2, users can create virtual machines with his preferred OS, disk size, virtual memory size, firewall etc. Thereafter we can access those machines just as we access normal physical machines in a remote location.  We can stop or terminate the machines when we are done with it or restart later when we need them again. In addition a snapshot of the machine can be created at any point so we can boot up another machines from the same state later. Following is a screenshot of the management console of EC2.

amazon-web-services

There are more services such as load balancing, scaling up and down etc. based on the user needs. Users are charged according to the resources they consumed such as virtual memory, storage and the time the machines were active. Another advantage of having an IAAS is we can scale up the system instantly.  We can add and remove more hard disk space, memory or machines themselves or database instances easily; no ordering, no deployment, no baseline etc. Just play without even plugging. Coolest thing is that resources can be added in rush hours and removed when things go dull. Dynamic resource management to its max! These systems also support automatic scaling up and scaling down so application developers don’t have to think of typical constraints of the shared computer architecture.

SAAS – Software as a Service

Adding another layer on top of the computing infrastructure, users are given a valuable service through a single piece of software or a collection of software. Google’s collection of SAAS is the most famous free services stack among the general public. Gmail and Google Docs are the popular example applications for free SAASs. Even this article itself was started in the clouds as a Google Doc first – in favor of cloud computing. There are thousands of other paid services which provides services like online time management systems, online POS systems etc. Paid services charge in different ways like per year, per user and PAYG. PAYG or pay as you go is the cost model that charge for the resources (computing power, storage etc.) consumed. Users have the freedom to choose the cost model that best suits them.  So you might question how free services generate revenue. The answer is it depends. Let’s consider Gmail as an example. Their main revenue model is advertisements.  Gmail has more than 500 millions of users which imply Gmail make more than 5 billion advertisements display if we assume one average user sees 10 advertisements a day. Although it is impossible to assume how much they make for a year, but it should be a plenty.  Additionally Google offers only a limited email and document space. Normal users can live with that. However for large corporate users needs to pay for additional email and document space when they grow. For e.g. there are numerous organizations in Sri Lanka uses Google services by paying subscription fees.

PAAS – Platform as a Service

There are organizations that provide platforms to build applications in the cloud where other organizations can build software services and host them. PAAS facilitates building of custom applications using a bunch of software platforms. This includes facilitating all of the development, maintenance and retiring stages of a typical software lifecycle. WSO2 Stratos,WSO2 App Factory, HerukoRedhat OpenShift are high standard PAAS vendors. Please read more about WSO2 App Factory in this introductory article by the same author.

As an example, WSO2 Stratos is a 100% open source, multi tenanted for private, public and hybrid cloud deployments. It provides a complete stack of middleware products as a service and also mediation, governance, security, gadgets, monitoring and more by using the capabilities of WSO2 products.  Third party runtimes such as PHP, MySQL, and Tomcat etc. can be plugged in via a concept called cartridge.  StratosLive is the Stratos version hosted by WSO2. It is free for anyone interested.

CLOUD FEATURES

Every service hosted in the Internet is not a cloud service. They have some common characteristics which qualifies a service as a cloud service which is listed below.

Multi tenancy

Single software is shared among multiple users or organizations called tenants. The system and the data are partitioned among tenants. Different tenants can have different configurations. But the software service provided should be the same with different features on or off based on costing model etc.

Auto Scaling

The system automatically scale up (spawn up serving instances) during the rush time and automatically scale down (kill additional services) during the free time.

Distributed

System supports multiple nodes/instances running concurrently in different machines in a network, ability to save and transfer the overall state to work as a single servicing entity is also an attractive feature.

 

 

cloud-people1

DISADVANTAGES OF THE CLOUD

As every coin has two sides, cloud computing also has some inherent disadvantages. Organizations which handle the confidential data such as government organizations, mission critical applications etc. may think twice before storing their information in a public cloud.

Though it seems to be cheaper at the beginning, cloud may be expensive in the long run. Another problem is organizations may have to tolerate cloud services providers downtimes and bugs which can hinder the overall quality of service. If you can remember recent service degradations of Outlook and Gmail, you know what I meant by that.

When looking at the current industry, more and more organizations tend to move to the cloud since organizations needs to get the competitive advantages by delivering their services or products early to the market. Almost all the popular companies like Microsoft, Amazon, eBay, IBM, eBay, Redhat etc. have stepped into the cloud. Big companies like Google are born due to the cloud computing, and all its services are available via cloud. Through cloud, companies can make their services available to the customer faster. Cloud native features help to build a more distributive, auto scaling services that that supports any number of users. Cloud let the companies to address more users.  For instance Microsoft Office what was a desktop application became more collaborative and accessible when it reached the cloud as Office 365.

Cloud is a new computing model of the era to build the applications shortly. However moving to a cloud might be time consuming initially as information may have to be separated and treated differently based on the security needs.

THE FUTURE

Rise of cloud computing is remarkable as it has made great success for this short time period. Almost every person who uses a computer or modern mobile phone, benefit from the power of cloud. Google, Email, Facebook, Dropbox, Google Drive, Skydrive are few of the system an average person uses which uses the Cloud. According to a recent report by IBM states that the businesses that moved to use cloud have doubled the revenue compared to its non-cloud counterparts. According a survey, 75 % of the surveyed businesses uses the cloud and the global market for the cloud is $158.8 Billion by 2014 which is 126.5% growth compared to 2013. So it is clear that the past of cloud computing was bright, but the future of cloud computing will be even brighter and business will be forced to use it to survive in the industry.

References

http://pzf.fremantle.org/2010/05/cloud-native.html?view=timeslide

http://whatis.techtarget.com/definition/multi-tenancy

http://www.gxsblogs.com/radkoj/2008/03/multi-tenancy-absolutely-critical-to-software-as-a-service-saas.html

http://www.businessinsider.com/the-10-most-important-companies-in-cloud-computing-2012-4?op=1

Image credits : creatiwittyblog.com, insideout.com, rent-a-techie.com/

By definition WSO2 App Factory is a multi-tenant, elastic and self-service Enterprise DevOps platform that enables multiple project teams to collaboratively create, run and manage enterprise applications.   Oh! kind of confusing? Yes, as most other definitions, only a few will grab what App Factory means from the first look at its definition. If it’s explained it in simpler words, WSO2 App Factory is a Platform as a service (PAAS) which manage enterprise application development from the cradle of the application to the grave. (Still confusing…? Figure below illustrates the move from the traditional on-premise software to cloud based services. You can see the Platform as a service in the third column.)

paas_illustration

Unless it is a university assignment or test, every real world application development has to undergo several phases until it is ready to go live. Applications has to be designed, developed and sent to QA for testing. Then, QA has to test them rigorously before approving for production. Then the bug fixing and stabilization phase. When the software is ready, it gets deployed. Finally when the application completed its job, it is needed to be retired.

Organizations have to use a number of tools in each of the above phases. For instance, developers may be using SVN for creating code repositories, maven or ant for building the projects, JIRA for ticket tracking and various other tools for finding bugs in the application. Above tools are independent of each other which results in organizations having to put a considerable effort in deploying those tools. If you are a developer, QA manager, system administrator or a DevOps or any other stakeholder who is involved in application development, there is no doubt that you have endured the pain of above and you might be wondering “Is there one single tool which does the work of all of the above tools?”. WSO2 App Factory does exactly that. By using App Factory you gain all the support for your application development, all under one roof.

Individual building blocks of the App Factory is illustrated in the below diagram.

wso2_appfactory-topology

Diagram 1 depicts the components of the App factory. Management portal, what is the main interaction point to the system is at the center. Source code management, issue trackers and other features are accessible via the portal. When a developer created an app via the management portal, he is provided with a space in the repo , space in the build environment and a project  in the issue tracker and so on. You clone from the repository you are provided into your development machine. Then develop the application with your favorite programming IDE and commit. WSO2 is planning to rollout a browser based IDE in the future to make the complete lifecycle run on the cloud. The application you are developing is continuously built in the cloud using your built tool. If automatic build is enabled, the build process will be triggered automatically when you commit. If auto deploy is enabled, the app will be deployed in the development cloud automatically after the build. Then after the development is completed, the apps will be promoted to the test cloud.  This promotion will retire the apps from the development cloud and deploy them in the test cloud. QA department will test them, promote to the production or staging cloud if tests pass or demote again to the development cloud if fail. The ultimate step is to send the apps to the app store enabling users to discover the apps. The most interesting thing is, all the above tasks can be executed using a single tool via a single management portal.

wso2_appfactory-lifecycle

 

Features of App factory

    1. Self-Provisioning of the workspace and resources such as code repository, issue tracking, build configuration and bug finding tools… etc.
    2. Support a variety of application

○     Web applications
○     PHP
○     Jaxrs
○     Jaxws
○     Jaggery
○     WSO2 ESB
○     WSO2 BPEL
○     WSO2 Data services

  1. Gather developers, QAs and DevOps of the organization to the application workspace
  2. Automate continuous builds, continuous tests and development activities
  3. One click solutions for branching and versioning
  4. Deploy application into WSO2 rich middleware stack
  5. No need to change your way of doing things

○     App factory can be configured to integrate with your existing software development life cycle.
○     Integrate with your existing users via LDAP or Microsoft Active directory

WSO2-AppFactory-applications-integrated-300x210

Yes, WSO2 App Factory is customizable. For instance organizations are not required to use the tools that App factory supports, they can plug in a tool of their preference. It is a matter of integrating another tool. Different organizations have different workflows, still App Factory can be configured to suit their own workflows.

In summary WSO2 App Factory is a cloud enabled DevOps PAAS for enterprise which manages the entire life cycle of an application. It leverages the application development giving enterprises a competitive advantage in the cloud.

Enough of talking, so help yourself by visiting App Factory preview in live. It is free and open source.

This article is just a bird’s eye view of the WSO2 App Factory. Visit its home page to broaden your knowledge. Good short video about the product is shown below: