Working with Virtual Servers on Talend Administration Center (TAC)

 

Working with Virtual Servers on Talend Administration Center (TAC)

Virtual Server Configuration :

A virtual server is a group of physical servers from which the best rated server will automatically get preferred at Job execution time. In Talend Administration Center, simply select physical servers and group them into a virtual server. Then set the execution task onto this virtual server as for any execution server, in order to automatically select the best server to execute the task onto.
Accessing the list of virtual servers
To access the list of virtual servers from Talend Administration Center as well as the execution servers that are assigned accordingly:
In the Menu tree view, expand Conductor and then click Virtual Servers to access the list of virtual servers.

The Virtual servers page is divided into two views:
• The Servers view on the left which regroups the physical servers you have configured in the Servers page. These execution servers can be either JobServers or Talend Runtime servers.
• The Virtual servers on the right which regroups the virtual servers and their assignments.
When you access this list for the first time, the existing physical servers are displayed on the view, but no virtual server is available.
From this page, you can create, edit and delete virtual servers and assign physical servers to them.

Adding a virtual server:

To add a virtual server from the Talend Administration Center home page:
1. From the toolbar on the Virtual servers view of the Virtual servers page, click Add a virtual server to open the [Virtual server] dialog box.

 

Add Virtual Server

 

Note: Cellar cluster: (Optional) Name of the cluster group.
Virtual servers without Cellar (Option 1): the features are deployed on all physical servers that are assigned to the virtual server.
Virtual servers with Cellar (Option 2): the features are deployed on one of the physical servers of the Cellar group that are assigned to the virtual server. Cellar then distributes the features to all runtime containers that have the same group, and ensures failover between servers.

Assigning one or several physical server(s) to a virtual server:

To group various execution servers in one virtual server, read the procedure below.
If you want to add clustered Runtime servers to your virtual servers, you must have previously installed clustered Runtime containers and created cluster groups.
1. On the Virtual servers page, select one or more server(s) in the Job servers view.
2. Drag and drop the selected server(s) onto a virtual server where you want to regroup or add the server(s).

 

select job servers

Failover and Load balance approach:

Failover: Failover redirects requests to alternate servers if the originally requested server is unavailable or too slow.

Load Balance:  Load balancing distributes request processing (work load) across multiple servers

Talend provides an alternate approach to handle this scenario using Talend Administration Center web application.

Load balancing process can be achieved using virtual servers concept and in order to handle this issue we need to configure virtual server as execution server in the task execution section.

execution server list

 

 

In the same way, Failover process can be achieved using ‘On unavailable JobServer’ option in the task execution window.

failover actions

 

This option allows you to decide what action you want to take on the selected task when the server is unavailable during a predetermined period of time (defined in the section called “Adding an execution server”).

Select one of the following from the list:

Wait: if the Job has an unknown status, usually caused by server problems, this option puts the task on hold throughout the period of time during which the server is inaccessible. When the server is accessible again, the task will restore the status it held prior to encountering the problem.

Reset task: if the Job has an unknown status, usually caused by server problems, this option puts the task status back to Ready to run when the predetermined period of time relating to the Job status elapses. However, the Job itself could continue running depending on the initial problem.

– Restart task: if the Job has an unknown status, usually caused by server problems, this option will automatically start the task in its original state when the predetermined period of time relating to the Job status elapses.

Note: The Restart task option is only useful when you are using virtual servers.

 – Recover task: if the Job has an unknown status, usually caused by server problems, this option will automatically recover the Job execution from the last validated checkpoint.

Note: The context parameters used when the task is reset, restarted or recovered, are the same as those used during the last execution

We can test both failover and load balancing process by scheduling multiple jobs on virtual servers or any one of the physical server.

 

Talend Administration Center Configuration

Talend Administration Center Configuration

Talend Administration Center is a Web-based administration application that allows Talend Studio project managers to administrate users and projects and manage access to the remote repository.

If you want to refer to my previous post on Talend Enterprise Studio for DI installation using Installer then click here.

Configuring the web application access:

The recommended way to configure the connection to the database and to the shared repository (SVN) is through the Web interface of Talend Administration Center.

  1. Start the application server (Tomcat or JBoss), then open a Web browser and type in the following URL: http://localhost:8080/<ApplicationPath>.
  2. Replace <localhost> with the IP address or the hostname of the Web server if the Web browser IP is different from the machine you are on and<ApplicationPath> with the Talend Administration Center Web application path.

For example, http://localhost:8080/org.talend.administrator.

  1. When connecting for the first time, a dialog box is displayed, asking you the password to access the Database Configuration page:

 

Type in the default admin password.

  1. H2 database connection parameters display and some automatic checks are performed on driver, url, connection, version information.

Note:

The administration database (storing users, rights, etc.) being an H2 embedded database, its access information is automatically set. However, if you do not want to use the embedded H2 database, you can set up a database server (MySQL, MSSQL or Oracle) in which case, set the required parameters.

  1. Click Set new license.
  2. Browse your system to the License file you received from Talend and click Upload. A final License check is performed.
  3. Click Go to Login.
  4. On the Login page, type in the default connection login for your first access:
  5. Default login: [email protected],
  6. Default password: admin.
  7. Those credentials correspond to the default user of the Web application. You can create a new one via the Users menu in Talend Administration Center, and then delete the [email protected] user after you are connected using the newly created user login.

Note:

If your Web access is restricted, you may need to click Validate your license manually to perform the validation of your license key. Follow the instructions on screen.

  1. Once the license is validated, the navigation bar of Talend Administration Center opens with all the pages accessible for the default administrator user account. For more information on which pages of Talend Administration Center an administrator user can access, see the Talend Administration Center User Guide.

You can now configure the link to the shared repository (SVN):

  1. Click Configuration to access the setting page of Talend Administration Center.
  2. Change the following parameters for the SVN module using the parameters you have set during the installation process of the SVN server.

– Server Location URL: your <repotis> SVN URL.

– Username: your <repotis> user.

– Password: your <repotis> SVN password.

The link to the SVN is now established, you can thus create a new project in order for the Talend clients to have at least one project in their workspace. To do so, you have to:

1. Create one or more users from the Users page;

2. Create a new, remote, collaborative project from the Projects page;

3. Associate the user(s) with the project from the Project authorizations page.

 

Accessing the Administration Center:

To access the Administration Center using the default Administrator account provided, do the following:

  1. In the address bar of your Internet browser, type in or paste the URL address corresponding to the location of Talend Administration Center.

For example, http://localhost:8080/org.talend.administrator

  1. Press Enter to display Talend Administration Center as shown in the below pic.

tac_login_page

Use below credentials to login into TAC.

Login: [email protected]

Password: admin

 

Database Configuration page:

  1. When you access Talend Administration Center for the first time, the Database Configuration page appears. Type in the administrator password (by default, it is admin) and click OK.

database_confi

2.Upon validation of your password, Talend Administration Center runs a series of checks. If no license or an invalid license is found, you will be prompted to specify a license. Click the Set new license button.

db_config_validation

3.  Click Browse to browse to your license file and click Upload.

license_browse

4.  Upon validation of your license, Talend Administration Center runs a series of checks again, and displays the following options:

    • Set new license: allows you to set a new license by repeating the previous step.
    • Validate your license manually: allow you to validate the loaded license.

 Go to login page: allows to you to open the login page to log in to the Talend Administration Center.

  1. Click Go to login page and type in the login and password provided by Talend in the [Login] dialog box.

talend_admin_login

  1. Click the Login button.

Talend Administration Center opens up on a welcome page.

 

Accessing the Configuration page:

The Configuration page offers accurate details on application parameters and their related values including database connection. All these parameters are grouped by module. This helps understand and troubleshoot module-related problems.

To access the Configuration page, click Configuration in the Menu tree view. The figure below illustrates an example.

tac_config_window

The second column in the configuration tabular list indicates whether a parameter is correctly set up  or empty or wrong .

The same icons appear as well in front of the group titles. They indicate whether the corresponding module is up and running and well configured  or if it contains any wrong parameter or is simply down .

 

Configuring TAC parameters according to our system:

Setting up Audit parameters:

audit_params

Setting up command line parameters:

              cmdline_param

Setting up monitoring (AMC) parameters:

amc_config

Setting up Job Conductor parameters:

job_conducter_config

Setting up Log4j Parameters:

log4j_config

Setting up Software Update parameters:

Before configuring this section, we need to start the artifact repository service depends upon the system environment like whether it is 32 bit or 64 bit.

Launching Archiva artifact repository:

For 32-bit Windows users:

  1. From a commandline window, browse to the bin directory of Artifact-Repository-Archiva-VA.B.C.

 

Ex: <Talend_installation_dir>/tac/Artifact-Repository-Archiva-V1.4.10.2/bin

 

  1. On 32-bit Windows, run the archiva.bat console command to run it interactively

For 64-bit Windows users:

Due to licensing constraints, The Archiva artifact repository distribution does not include Tanuki wrapper for 64 bit Java on Windows.

  1. Check that you are running 64-bit Java.
  2. Download 32-bit Java to a separate directory.
  3. Configure a batch file to set the JAVA_HOME variable to the 32-bit JDK,

Then run the archiva.bat console command from within the same batch file so that the JDK is pointing to the correct location.

After creating the new batch file add below parameters to it like below.

 

Archiva_32bit.bat file:

@echo off

set JAVA_HOME=E:\Program_Files\Java_32bit\jdk_1.7\bin\java

call archiva.bat console

This will provide a 32-Java environment just for the Archiva artifact repository, while the rest of your applications run safely in 64-bit mode.

 

Artifact Repository configuration

To configure Talend Software Update, open your Web browser and type in the URL to your instance of the Archiva artifact repository, http://localhost:8082/archiva for example, then proceed as follows:

  1. Log in with the following credentials:

Username: admin

Password: 1q2w3e4r

Once we have launched and configured the Software Update repository, go to the Configuration page of Talend Administration Center and fill in the following information in the Software Update group:

  • Repository URL: Type in the location URL to your software update repository, http://localhost:8082/archiva for example.
  • Admin username: Type in the name of the repository user with Manager role. By default, it is SoftwareUpdateAdmin.
  • Admin password: Type in the password of the repository user with Manager role. By default, it is SoftwareUpdateAdmin.
  • Reader username: Type in the name of the repository user with Observer role. By default, it is SoftwareUpdateReader.
  • Reader password: Type in the password of the repository user with Observer role. By default, it is SoftwareUpdateReader.

                    software_update_config

Setting up SVN parameters:

                    svn_config

Launching the Talend products for the first time:

We installed all the talend products via Talend Installer, so we are nothing to do as it has been unzipped in the Installer installation directory (called Talend by default) with its license file.

Below are the sub-folders available in the Talend installation directory and we can launch by starting their services.

— cmdline

— jobserver

— logserv

— studio

— tac

Launch studio with Remote connection:

Go to talend installation directory path <talend installation path>/studio and launch the appropriate executable file according to your system.

studio_svn

Go to connection section and click on the configuration button (…) to create/edit a connection from the drop down menu.

remote_connection_studio

Click on Ok and select the appropriate project from the svn repository to launch the studio.

Managing users:

Adding a user:

On the Users page, click Add to create a new account. You can create a Viewer, Designer, Operation Manager, Administrator or a user with several of these roles.

add_user

Managing projects:

Adding a project:

To add a project which is stored in SVN

  1. In the Menu tree-view, select Projects to open the project list page.
  2. From the toolbar, click Add. Mandatory fields in the Project panel to the right are followed by .

                         add_project

Managing project authorizations:

The Project Authorizations page in Talend Administration Center allows you to manage user authorizations. From this page, you can:

  • give a user the right to access one or more projects,
  • Specify if the access right is read-only or read-write.

 

Authorizing users:

To give a user the right to access one or more projects, do the following.

Note that the assigned users have to re-launch the Studio to take this authorization into account.

 

  1. In the Menu tree-view, select Project authorizations to display the authorization list.
  2. In the Project panel, select the project to which you want to assign a user.
  3. In the Right column of the User panel, give read or read and write permissions to a user by clicking the corresponding icons.

The Read and Read write icons show on the projects on which the user can work by default. If a user cannot work on a project, a prohibition icon indicates that you cannot give rights to this user: it depends on the Type that is defined during the user creation in the Users page.

The number of users that have read access as well as the number of users that have read and write access to a project are updated in the corresponding columns of the Project panel.

project_authorize_page

 

 

Executing Jobs from Jobs Conductor:

In the Job Conductor page of Talend Administration Center, an execution task gathers the script generation, deployment and execution phases of data integration Jobs. You can launch this task, from this single web-based application, using a simple or a CRON trigger.

Working with Job execution tasks:

A Job execution task represents a set of actions that you can configure in Talend Administration Center in order to execute the Jobs designed in Talend Studio directly from the Administration Center. You can also define different types of triggers to launch an execution task.

To access the execution task list, do the following:

In the Menu tree view, click Job Conductor to display the list of scheduled tasks that will deploy and execute Jobs on a remote server.

                         job_conductor_execution

Adding an execution task

The Talend Administration Center allows you to add execution tasks on Jobs that are either based on the SVN repository, or pre-generated by the Studio as zip files. According to your needs, read one of the procedures below.

Note that once the task is added, you still can switch from one mode to the other.

Adding an execution task on a Job based on SVN repository

From the toolbar on the Job Conductor page, click Add task to clear the Execution task configuration panel.

execution_task

Click on save button will create a execution task.

Once the execution task is created then we will able to deploy and execute the task from the same section by clicking on generate, deploy and run buttons accordingly.

 

Thanks,

Sayagoud Ravelly