job scheduling
Run Tasks On Your Schedule
Flux supports very detailed scheduling requirements:
Define Jobs Using Workflows
Using a workflow model, you can create jobs with all kinds of conditional dependencies, branching, looping, and parallelism. Workflows (state machines) are inherently more powerful than the traditional job scheduling model.
Parallelism and Timeouts
Create jobs that execute different paths of execution simultaneously. Create jobs that follow a timeout path in a workflow if an event does not occur soon enough or a task takes too long to execute.
Job Dependencies
Flux supports sophisticated job dependencies. You can run one job when a different job finishes or just reaches a certain milestone. Dependencies can be loosely-coupled and don't require you to write code to configure them. Due to Flux's powerful workflow model, you can define dependencies at any point within your job, not just at the starting point of a job.
View an Entire Job on One Screen
A Flux job can contain hundreds of steps and dependencies. You can view it all on a single screen in your web browser.
managed file transfer
Extensive FTP, SFTP, and FTPS Support
Flux supports file watching and file transfer jobs and workflows that use:
Watch for Inbound Files
Watch for incoming files that arrive via FTP, Secure FTP, FTP-SSL, the local file system, or a UNC network share. You can wait until the files arrive completely to avoid processing incompletely transferred files. Once your files arrive, process them within your job or workflow.
Streaming Server-to-Server File Copies
When copying files from FTP/SFTP/FTPS server to another, Flux streams the file so that it is written to disk only on the target FTP server. By not writing a temporary copy of the file, the file transfer is faster and does not occupy disk space on the intermediary server, where the Flux engine runs.
Issue FTP Commands Directly to Server
For issuing non-standard commands that only your FTP server supports or for fine-grained control over your FTP server, you can issue FTP commands directly to your FTP server within a job or workflow.
Change File Permissions
If your FTP server supports it, you can change file permissions on files on your FTP server. This chmod behavior is not part of the FTP specification and may work on one FTP server but not another.
AUTOMATED WORKFLOWS
POWERFUL STATE MACHINE
Define simple or intricate automated workflows using a powerful state machine model. Define workflow steps that with conditional branching, dependencies, looping, parallelism, splits, and joins. Perform tasks and wait for events to occur within your workflow.
scalability
Scale individual workflows to 500 steps. Run thousands of workflows across a cluster of Flux workflow engines.
error handling
Errors occur in workflows. When they do, you can define simple or detailed mechanisms to handle them. Handling errors can be fully automated or call out to people to take corrective actions.
DEPENDENCIES ON OTHER WORKFLOWS AND FILES ON FTP SERVERS
Create workflows that depend on other workflows. Wait for other workflows to reach milestones or to finish altogether. Create workflows that depend on and watch for files on FTP servers to arrive or change
WEB-BASED MONITORING
MONITOR EVERYTHING FROM YOUR WEB BROWSER
Using Flux's web-based Operations Console, watch over and monitor all your jobs, workflows, and file transfers. Designed for the needs of Operations Staff but usable by developers too.
NEAR REAL-TIME UPDATES
The Flux Operations Console uses automated, asynchronous, Ajax-style technology to keep your browser window updated with the status of all your Flux activities, automatically updated in near real-time.
WATCH FILE TRANSFERS IN REAL-TIME
Watch file transfers taking place from within jobs and workflows. Monitor file transfer statistics like source and destination hosts, file transfer protocol, transfer rate, and estimated time remaining.
MONITOR 500 000+ JOBS, WORKFLOWS, and FILE TRANSFERS SIMULTANEOUSLY
The Flux Operations Console can monitor 500 000+ jobs, workflows, and file transfers, all at the same time.
design jobs and workflows visually in the browser
Visual Design
Design your jobs and workflows using a drag-and-drop visual interface, right in your web browser. Integrated into the Flux Operations Console. No need to install a desktop client.
Drag and Drop
Design jobs and workflows using drag-and-drop. Right in your web browser.
See Your Job Dependencies
Look at your job dependencies and workflow dependencies, all within the browser.
Programming Not Required
To design multi-step jobs and workflows — complete with dependencies, conditional branching, and looping — you don't need to be a software developer. (But if you want to use Flux's full Java API, you can.)
error recovery via automated and manual means
WHEN SOMETHING GOES WRONG
React accordingly when a process errors out or a file transfer fails. Configure automated responses to errors or call out for human attention. Error handlers can be complex workflows in and of themselves.
AUTOMATED AND MANUAL ERROR RECOVERY
At any point in a job, define automated or manual error handlers to resolve, retry, or report the situation.
SOPHISTICATED ERROR HANDLING
Flux's error handlers can be defined in terms of a workflow, allowing you to create sophisticated error recovery mechanisms.
REUSE ERROR HANDLERS
Error handlers can be defined once and used throughout a single job or throughout an entire collection of jobs.
run processes on remote machines
AGENTS
Using a Flux Agent, run a process on a remote computer.
TAKE ADVANTAGE OF SPECIAL RESOURCES
Sometimes, a process needs to run on a specific computer that contains unique resources. Install a Flux Agent on that computer and run tasks on that computer using Flux.
POOLS
You can define a group of computers into a pool, and then schedule processes to run on any one of the machines in that pool.
AGENTLESS SCHEDULING
Sometimes, you don't want to install special agent software on your computers. Flux supports so-called agentless scheduling that uses SSH to run processes on remote machines.
automated database processes
CALL STORED PROCEDURES
From within your jobs and workflows, call stored procedures defined in local or remote databases. Pass data in to your stored procedures. Pass the results through to the next step in your workflow.
ISSUE QUERIES
Issue database queries against local or remote databases and process the results within your workflow.
PERFORM DATABASE UPDATES AND INSERTS
Update existing data or add new rows to your tables in your local and remote databases.
PASS JOB DATA TO DATABASE CALLS
Use the information retrieved or generated within your job or workflow and pass it through to your database calls.
RETRIEVE DATA AND PASS TO JOB
Query your databases and pass the data through to your jobs for processing.
invoke rest and soap web services
CALL REST SERVICES
Use Flux to invoke REST services from your jobs and workflows.
CALL WEB SERVICES
Similar in nature to REST services, Web services can be called from jobs and workflows.
PASS DATA IN AND OUT
Pass job data into REST and Web services. Pull data from REST and Web services into your jobs and workflows for processing.
SECURITY
Use the security mechanisms built into REST and Web services like SSL support, key stores, and trust stores.
embed in java se and java ee apps
EXTENSIVE JAVA API
Flux has a deep Java API for use in Java EE (J2EE) and Java SE (J2SE) applications. Just add the flux.jar file and any supporting jar files needed to your Java class path. All of Flux's functionality is available via its Java APIs so you can add job scheduling, workflow, and managed file transfer capabilities to your application.
APPLICATION SERVER SUPPORT
Flux supports popular application servers like WebLogic, WebSphere, JBoss, Tomcat, and Jetty. You can embed Flux and your application within these application servers.
DATA SOURCES, JMS, EJB
Flux supports the usual application server technologies like using data sources, publishing and receiving JMS messages, creating EJBs, and calling EJBs.
JAVA SE APPLICATIONS SUPPORTED, TOO
If you're not using an application server, that's fine. Flux fully supports running in Java SE environments, where all you have is the JVM, your application, flux.jar, and any supporting jar files.
scalability, high availability, and load balancing
CLUSTERING AND FAILOVER
To provide high availability and increased throughput, Flux can be clustered on two to eight servers. There is no single point of failure in the cluster. If one server goes down, that server's duties are automatically and seamlessly moved to the other servers in the cluster.
SCALE TO 30 000+ JOBS
Scale to thousands of jobs and workflows.
HIGH AVAILABILITY
There is no single point of failure. All servers in the cluster cooperate equally to increase the system's overall throughput even in the event of server failure.
LOAD BALANCING
Flux automatically spreads the task load out across all the servers in the cluster. Less busy servers receive new work before busier servers. In jobs and workflows that contain parallelism, different parts of the same workflow can run simultaneously on different servers in the cluster.
multi-platform and lightweight
MULTI-PLATFORM
Flux runs on a very wide variety of hardware and operating systems. Flux is developed in Java using a single code base. When a new Flux version is released, all features are available on all supported platforms.
MAINFRAMES, BIG IRON, PCs, LAPTOPS
Flux runs well on small computers up to big servers and mainframes. Flux customers run Flux on very wide variety on production hardware.
LIGHTWEIGHT
Flux does not consume much processor, memory, or hard drive space. Flux runs equally well on laptops, workstations, enterprise-class servers, and mainframes.
WINDOWS, LINUX, z/OS, WHATEVER
Flux runs on just about any operating system with a modern Java runtime environment — Windows, Linux, z/OS, Solaris, et cetera.
security
REGULAR USERS AND ADMINISTRATORS
You can configure Flux with different permissions to allow or restrict access to different jobs or different components of different jobs.
LDAP AND ACTIVE DIRECTORY
Flux can tap into your organization's LDAP and Active Directories for single sign-on purposes.
SECURITY IN OPERATIONS CONSOLE AND JAVA APIS
Flux's security mechanisms are visible in the Operations Console, the Java APIs, and the command line interface.
DISABLE SECURITY IF YOU WANT
Some environments, especially when Flux is embedded in Java applications, you don't want any kind of security. With a configuration option set appropriately, all Flux security is diabled.
reporting
KNOW WHAT JOB OR WORKFLOW RAN AND WHEN
Run reports to determine what tasks, steps, actions, jobs, and workflows executed within a specified time frame.
FILTERING
Filter reports on different criteria.
THREE KINDS OF REPORTS AVAILABLE
Flux includes three different kinds of reports.
SAVE REPORT TO FILE
Save your reports to a file on your local computer.
cloud computing
ADD AND REMOVE FLUX ENGINES DYNAMICALLY
You can configure a cluster of servers where Flux runs and add new Flux servers on the fly. Remove Flux servers dynamically. New Flux servers join your cloud automatically. Removed Flux servers gracefully fade away and leave the cloud intact.
THE CLUSTER IS YOUR CLOUD
With Flux, the cluster is the cloud. The cloud is the cluster. Your clients access your cloud and don't care whether new Flux servers have been added or removed.
INCREASE SCALABILITY AS NEEDED
In your cloud (cluster), add more Flux servers during periods of high demand.
FALLBACK AS NECESSARY
As your scalability and high availability needs decrease dynamically, remove Flux servers from your cloud (cluster). The remaining Flux servers continue running unaffected.
