Skip to content

Instantly share code, notes, and snippets.

@EvilFreelancer

EvilFreelancer/evaluation.frida-gguf.ipynb

Last active May 27, 2025 11:24
Show Gist options
  • Save EvilFreelancer/b2833527b040e7811e1842c47a09415d to your computer and use it in GitHub Desktop.
Save EvilFreelancer/b2833527b040e7811e1842c47a09415d to your computer and use it in GitHub Desktop.
Download ZIP
FRIDA GGUF тестирование
Display the source blob
Display the rendered blob
Raw
evaluation.frida-gguf.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Raw
results.md
Name Original GGUF f32 GGUF f16 GGUF q8_0 GGUF tq2_0 GGUF tq1_0
STSBTask 0.8444961710957753 0.7736137897844315 0.7736028484956837 0.773361958282741 0.3647773184116074 0.3647773184116074
ParaphraserTask 0.760147408288547 0.5760219715717907 0.5760429923959474 0.5761467826017955 0.11769859613134119 0.11769859613134119
XnliTask 0.4796407185628742 0.4165668662674651 0.4165668662674651 0.4155688622754491 0.3393213572854291 0.3393213572854291
SentimentTask 0.836 0.7983333333333333 0.7976666666666666 0.799 0.5176666666666667 0.5176666666666667
ToxicityTask 0.989866 0.982673 0.98267 0.982684 0.724437 0.724437
InappropriatenessTask 0.8463448532935726 0.8284147185686138 0.8285707439160764 0.828561158741147 0.5766776718675115 0.5766776718675115
IntentsTask 0.8034 0.7386 0.7386 0.7392 0.2868 0.2868
IntentsXTask 0.781 0.6956 0.6956 0.6942 0.1096 0.1096
FactRuTask 0.24240861548860063 0.12707425468979844 0.12693111870225876 0.12664414521133774 0.05964525242263313 0.05964525242263313
RudrTask 0.26938953157907136 0.15240762044883197 0.15240762044883197 0.1553384424039951 0.060060972146900204 0.060060972146900204
SpeedTask (gpu) 28.902501265207924 - - - - -
SpeedTask (cpu) 994.7337913513184 184.08727248509723 110.00351905822754 81.11035426457723 53.92512798309326 90.35333156585693
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment