Clay Icon

Are you able to run more than 4 concurrent runs with Apify?

·
·

Are you able to run more than 4 concurrent runs with Apify?

  • Avatar of kushagra
    kushagra
    ·
    ·

    yup are you hitting the memory usage quota in 4 runs? try reducing the memory requirement per run

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    it doesnt even come close to my memory limit

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    Will use 2/32gb with 4 runs

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    Mostly just trying to figure out if this is on Clay's side or Apify

  • Avatar of kushagra
    kushagra
    ·
    ·

    that's weird, i have done multiple runs concurrently in the past and the times it wasn't working was because of the memory limit on apify side (see image)

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    It's not that it doesn't work or gives an error, it just wont run more than 4 at a time when I run a column in clay

  • Avatar of kushagra
    kushagra
    ·
    ·

    that def doesn't sound right, should checkin with support which actor are you using, i can give it a try to see if it's the same for me

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    In Clay the entire column will say queued, but in Apify it only shows 4 runs at a time

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    This is true of other actors as well though

  • Avatar of kushagra
    kushagra
    ·
    ·
  • Avatar of kushagra
    kushagra
    ·
    ·

    if you're on a free plan - that most likely is the issue because of these limits

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    Yea I'm not, my max memory is 32gb

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    Also if I manually start runs from within the console I can do a lot more than 4

  • Avatar of kushagra
    kushagra
    ·
    ·

    damn, asking apify let's see what they have to say, my hunch is some api limit on their end

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    lmk if you hear anything back here, chatting with both Clay and Apify support and both are saying it's on the other end

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    btw if it helps, it's still capped at 4 concurrent runs even across multiple actors (when I run 2 separate clay tables that use different actors, its still 4 total)

  • Avatar of kushagra
    kushagra
    ·
    ·

    nope didn't hear anything from them yet - you might want to just try a HTTP API column to run your actor that will rule out any issue on Clay's side ie if you still get 4 runs using HTTP API the issues is for sure not on Clay's side

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    Do you use Apify through the integration or HTTP?

  • Avatar of kushagra
    kushagra
    ·
    ·

    i use it through the integration but you can run it via the api too as a check

  • Avatar of kushagra
    kushagra
    ·
    ·

    i have done it in the past to pull results and automate a workflow

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    and if you run a column, how many run concurrently for you?

  • Avatar of Jack M.
    Jack M.
    ·
    ·

    through the integration

  • Avatar of kushagra
    kushagra
    ·
    ·

    haven't checked that lately - but don't remember runs being a limiting factor for me in the past