← Back to Home

Help & Documentation

Everything you need to get started and get the most out of Data Shepherd.

Getting Started

Your first transformation

  1. Go to Transformations → New Transformation
  2. Upload your source file — CSV, JSON, EDI, TXT, or fixed-width. If you have a spec document (PDF, Word) that describes the file format, upload it as a Data Dictionary for better results.
  3. Optionally upload a target file if you need to map one format to another.
  4. Choose Quick Mode for a single prompt, or Chat Mode to refine the transformation conversationally.
  5. Preview the output on your sample data. If something is off, tell the agent what to fix.
  6. Save the transformation. It's now ready to run on any file with the same structure.

Running a saved transformation

Open any saved transformation and click Run. Upload the file you want to transform, and Data Shepherd will execute the scripts and return the result. You can download the output or view it in the browser.

Same transformation, same output, every time — the scripts are deterministic, so you get consistent results regardless of when you run them.

Supported File Formats

CSV / TSV — comma or tab-delimited
JSON — objects, arrays, nested structures
TXT — delimited text files
Fixed-width — positional column layouts
EDI — X12, HL7, and other interchange formats

You can also upload PDF, Word, and Excel files as data dictionaries or supplemental context — these help the AI understand your file format and produce more accurate results.

Using the Agent Chat

Chat Mode lets you build transformations conversationally. Describe what you need in plain English, and the agent will analyze your file, propose a transformation plan, and generate the scripts.

Tips for better results

  • Be specific about field names and output format — "combine first_name and last_name into full_name" beats "merge the name columns"
  • Upload a data dictionary if you have one. Spec documents, PDFs, and Word files all work.
  • If the preview isn't right, tell the agent what's wrong. It will revise and show you a new preview.
  • You can upload additional files mid-conversation if the agent needs more context.
  • For complex mappings, upload both a source and target sample so the agent can see exactly what you need.

What the agent will ask you to approve

  • Data dictionaries — the agent extracts field definitions from your documents and asks you to confirm before proceeding.
  • Transformation plans — before generating scripts, the agent shows you what it intends to do. Approve or request changes.

How Your Data Stays Private

Data Shepherd is designed so that AI never processes your actual data records.

  • AI sees structure, not content. The AI analyzes a small sample to understand field names, formats, and types. Your full dataset is never sent to the AI model.
  • Scripts are deterministic. Once generated, the scripts run server-side without AI involvement. Same input, same output, every time.
  • You approve before anything runs. Preview the output on sample data first. Nothing touches your full dataset until you say so.
  • Data stays on the platform. Files are processed in isolated server environments and are never shared with third parties.

Connectors & Schedules

Connectors

Connectors link Data Shepherd to your cloud storage so transformations can run automatically. Supported types:

  • SFTP — authenticate with password or SSH private key
  • FTPS — FTP over SSL/TLS with password authentication
  • Amazon S3 — authenticate with access key and secret
  • Azure Blob Storage — authenticate with connection string or SAS token

You can test connectivity before saving. Credentials are stored securely and never exposed in the UI after creation.

Schedules

Schedules run transformations automatically on a recurring basis. Connect a source (where files arrive), a transformation (what to do with them), and optionally a destination (where to send the output).

  • Set a file pattern (e.g., *.csv or report_*.json) to pick up specific files.
  • Choose what happens after pickup — archive, move, delete, or leave in place.
  • Skip duplicates to avoid reprocessing files you've already handled.
  • Pro and Max plans support encryption (PGP/GPG or AES-256) for both source decryption and destination encryption.

Auto-Heal

Source data changes — columns get renamed, formats shift, new fields appear. Auto-heal detects when a saved transformation fails on new data and automatically generates revised scripts to handle the changes.

  • Enable or disable auto-heal per transformation from its detail page.
  • When a job is auto-healed, you'll see a summary of what changed and can choose to accept the updated version.
  • Available on Pro and Max plans.

API & Webhooks

Integrate Data Shepherd into your own systems with the REST API. Run transformations, check job results, and monitor webhooks programmatically.

  • Create API keys in Settings → API Keys & Webhooks with scoped permissions (read, run).
  • API keys are for running transformations and monitoring results. Creating and managing transformations, schedules, and connectors is done on the site.
  • Set up webhooks on the site to get notified when jobs complete or fail. Monitor webhook deliveries via the API.
  • API documentation is available in the API Keys & Webhooks settings tab.
  • Available on Growth, Pro, and Max plans.

Account & Billing

  • Upgrade or downgrade anytime from Settings → Subscription. Changes are prorated.
  • Manage billing (update card, download invoices) through the Stripe billing portal, accessible from Settings.
  • View usage on your Dashboard — saved transformations, jobs this period, and generations remaining.
  • Email notifications can be configured in Settings → Notifications for job completions, failures, and weekly summaries.
  • Delete your account from Settings → Account. This permanently removes all transformations, job history, and subscription data.

Plan Limits

FreeGrowthProMax
Price$0$19/mo$59/mo$249/mo
Transformations31550200
Runs / month2575150Unlimited
Max file size5 MB25 MB50 MB75 MB
API accessYesYesYes
Connectors15Unlimited
Schedules110Unlimited
Auto-healYesYes
EncryptionYesYes

Annual billing saves approximately 17%. See Pricing for full details.

Need Help?

If you run into something that isn't covered here, reach out at support@datashepherd.ai. We typically respond within one business day.