APIEval-20: A Benchmark for Black-Box API Test Suite Generation

 (huggingface.co)

4 points | by AkshatVirmani 4 hours ago

3 comments

  • riyajoshi 4 hours ago
    Nice to see a benchmark in this space especially with black-box constraints.
  • akshay_93 4 hours ago
    like that the scoring bias is toward bug detection & not test generation only. generating lots of tests with AI is easy but that doesn't necessarily mean they're good
  • saikia_ 4 hours ago
    curious.. let me see if this works for our internal setup