With a few key things and an FTP client like WinSCP, you can get better automation and improve your file transfer processes.
WinSCP is a popular Windows file transfer client. We often get asked about it’s capabilities and compatibility with our cloud FTP service. One of reasons for it’s popularity is the flexibility and power of its scripting and automation cabapilities.
When you install WinSCP, two executables get added to your computer — WinSCP.exe, which presents the familiar windowed interface for interactive transfers, and WinSCP.com, the command-line client.
It’s worth your time to check out WinSCP’s guide for creating automation using WinSCP.com, including useful example scripts.
This document won’t cover how to use WinSCP.com for scripting. Instead, we’ll discuss some things to keep in mind as you create your own automation scripts: logs, SSH keys, and planning.
1. Logs, Logs, Logs
The WinSCP.com command-line client can automatically create log files. Logs are invaluable when something goes wrong — you’ll see exactly which command failed and what error messages were included. If you know when the script fails, that helps point you at what to fix.
WinSCP.com allows you to control the name and format of your log files. You can also automatically configure log rotation by setting the maximum log size along with the maximum number of archive logs to keep. See more details about how to do this at https://winscp.net/eng/docs/commandline#logging
2. Configure (& Use) SSH Keys Rather Than Passwords
For SFTP connections, you can choose to connect to ExaVault with an SSH key. This type of connection is advantageous when you’ve created a script to automate your transfers. Instead of putting your password in your script, you can instead provide an SSH key.
ExaVault provides an easy-to-use interface for creating SSH keys and assigning them to specific users. Check out our documentation at https://www.exavault.com/docs/account/04-users/04-ssh-keys
3. Plan Ahead for Failure
Sometimes, your script will fail for unexpected reasons. Here are some possible scenarios:
- You can’t upload any new files because your account space is filled up
- Your internet connection is offline when your task is scheduled to run
- The files you want to download aren’t on the server right now
- Someone has changed your user’s password (another argument for SSH keys)
- Your account has exceeded the daily transaction limits, and logins are failing
- The WinSCP.com process crashes when it runs out of memory
These are just a few problems that can prevent your FTP automation from finishing successfully. If you are clever, you may find others. It’s not a question of if your script will fail, but what you will do when that happens.
Make a plan for detecting failures. How will you know something went wrong with your automation (preferably before your customers or vendors see any impact)?
A few tools available to help:
- Notification emails are a good confirmation that expected actions happened and are helpful for figuring out when something stopped working if not noticed right away.
- Using a batch file to run the WinSCP.com executable will allow you to check the returned %ERRORLEVEL% of the WinSCP.com program and potentially send off an alert that something went wrong.
- Remember that first thing? Logs, Logs, Logs! Using log files with your automation provides a quick way to check for problems — often they’ll be spelled out right in those logs.
Once you’re aware that your transfers didn’t complete the way you expected or how you expected, you should also plan for how you will recover.
Make sure to ask yourself several questions as you plan ahead.
- Are you able to re-run the script whenever you like, or is the automation locked away on a scheduling system you don’t have access to?
- Will the system automatically recover if the script is successful next time, and is that an acceptable delay?
- What needs to happen to make sure everything is working properly?
Successful Automations
WinSCP is a powerful and popular tool for automating file transfers. Combining smart usage of log files and SSH keys with a detailed plan can make the difference between an automation that saves you time and a frustrating emergency.
Originally published at https://www.exavault.com on June 23, 2021.
