Help
RSS
API
Feed
Maltego
Contact
Domain > fyrd.science
×
Welcome!
Right click nodes and scroll the mouse to navigate the graph.
×
More information on this domain is in
AlienVault OTX
Is this malicious?
Yes
No
DNS Resolutions
Date
IP Address
2019-09-15
104.18.46.246
(
ClassC
)
2024-09-18
172.67.160.121
(
ClassC
)
Port 80
HTTP/1.1 301 Moved PermanentlyDate: Wed, 18 Sep 2024 05:46:45 GMTContent-Type: text/htmlContent-Length: 167Connection: keep-aliveCache-Control: max-age3600Expires: Wed, 18 Sep 2024 06:46:45 GMTLocation: https://fyrd.science/Report-To: {endpoints:{url:https:\/\/a.nel.cloudflare.com\/report\/v4?sJtirH%2FNJUhp6z6yIyVX45K8BIi5fpxunSGwl%2FtOS2e7MDeDISawxL8DIDzPCxICimNB20yGoSy54dPl8u2FKtEHRsB7gQi6gVE%2FdRQEvLOEooPY3SRE9l6wJzP6GZ7U%3D},group:cf-nel,max_age:604800}NEL: {success_fraction:0,report_to:cf-nel,max_age:604800}X-Content-Type-Options: nosniffServer: cloudflareCF-RAY: 8c4f09913b3127ad-SEA html>head>title>301 Moved Permanently/title>/head>body>center>h1>301 Moved Permanently/h1>/center>hr>center>cloudflare/center>/body>/html>
Port 443
HTTP/1.1 200 OKDate: Wed, 18 Sep 2024 05:46:45 GMTContent-Type: text/html; charsetutf-8Transfer-Encoding: chunkedConnection: keep-alivelast-modified: Wed, 07 Mar 2018 00:31:35 GMTvary: Accept-Encodingaccess-control-allow-origin: *expires: Wed, 18 Sep 2024 05:56:45 GMTCache-Control: max-age600x-proxy-cache: MISSx-github-request-id: 7352:6735E:22868A0:26AB1F4:66EA6945CF-Cache-Status: DYNAMICReport-To: {endpoints:{url:https:\/\/a.nel.cloudflare.com\/report\/v4?sExNNykJ5OPVYnzyLuUVvB9Sol8gbOuD7iNUfFdO%2BrBxJ2irltSsVgbvZo9i0fUTX9zHR3heiMCcCiss6QU%2BJDEbh54hB1WkkgfUSIYkJiN%2F9vTXzCuFtbrIGtdfXoDc%3D},group:cf-nel,max_age:604800}NEL: {success_fraction:0,report_to:cf-nel,max_age:604800}Strict-Transport-Security: max-age15552000; preloadX-Content-Type-Options: nosniffServer: cloudflareCF-RAY: 8c4f0991ab713078-SEAalt-svc: h3:443; ma86400 !DOCTYPE html>html langen-us>head>meta charsetUTF-8>title>Fyrd/title>meta namedescription contentFyrd Python Cluster Manager: A python cluster: submit functions and shell scripts to torque, slurm, or local clusters/machines using easy python sytanx. Compatible with Python 2.7+.>meta namekeywords contentpython cluster, python torque, python slurm, python multiprocessing, torque, slurm>meta nameauthor contentMike Dacre>meta nameviewport contentwidthdevice-width, initial-scale1>link relsitemap typeapplication/xml titleSitemap hrefhttps://fyrd.science/sitemap.xml />link relauthor hrefhttps://www.michaeldacre.com />link relpublisher hrefhttps://www.michaeldacre.com />link relstylesheet hrefhttps://fyrd.science/css/normalize.css>link hrefhttps://fonts.googleapis.com/css?familyOpen+Sans:400,700 relstylesheet typetext/css>link relstylesheet hrefhttps://fyrd.science/css/cayman.css>link relstylesheet hrefhttps://fyrd.science/css/custom.css>link relstylesheet hrefhttps://fyrd.science/css/syntax/default.css>link relapple-touch-icon sizes180x180 hrefhttps://fyrd.science/assets/logo/favicon/apple-touch-icon.png?v1>link relicon typeimage/png hrefhttps://fyrd.science/assets/logo/favicon/favicon-32x32.png?v1 sizes32x32>link relicon typeimage/png hrefhttps://fyrd.science/assets/logo/favicon/favicon-194x194.png?v1 sizes194x194>link relicon typeimage/png hrefhttps://fyrd.science/assets/logo/favicon/android-chrome-192x192.png?v1 sizes192x192>link relicon typeimage/png hrefhttps://fyrd.science/assets/logo/favicon/favicon-16x16.png?v1 sizes16x16>link relmanifest hrefhttps://fyrd.science/assets/logo/favicon/manifest.json?v1>link relmask-icon hrefhttps://fyrd.science/assets/logo/favicon/safari-pinned-tab.svg?v1 color#5bbad5>link relshortcut icon hrefhttps://fyrd.science/assets/logo/favicon/favicon.ico?v1>meta namemsapplication-config contenthttps://fyrd.science/assets/logo/favicon/browserconfig.xml?v1>meta nametheme-color content#ffffff>script typeapplication/ld+json>{ @context: http://schema.org, @type: SoftwareApplication, name: Fyrd, operatingSystem: Linux, OS X, applicationCategory: Library,}/script>script srchttps://code.jquery.com/jquery-3.1.1.min.js type227bee229a7529d611148ae2-text/javascript>/script>script srchttps://cdnjs.cloudflare.com/ajax/libs/jquery.sticky/1.0.3/jquery.sticky.min.js type227bee229a7529d611148ae2-text/javascript>/script>script type227bee229a7529d611148ae2-text/javascript> $(document).ready(function(){ $(#menu).sticky({topSpacing:10}); }); /script>script srchttps://cdn.jsdelivr.net/jquery.scrollup/2.4.0/jquery.scrollUp.min.js type227bee229a7529d611148ae2-text/javascript>/script>script type227bee229a7529d611148ae2-text/javascript> $(function () { $.scrollUp({ scrollName: scrollUp, // Element ID scrollDistance: 600, // Distance from top/bottom before showing element (px) scrollFrom: top, // top or bottom scrollSpeed: 300, // Speed back to top (ms) easingType: linear, // Scroll to top easing (see http://easings.net/) animation: fade, // Fade, slide, none animationSpeed: 200, // Animation speed (ms) scrollTrigger: false, // Set a custom triggering element. Can be an HTML string or jQuery object scrollTarget: false, // Set a custom target element for scrolling to. Can be element or number scrollText: Scroll to top, // Text for element, can contain HTML scrollTitle: false, // Set a custom a> title if required. scrollImg: false, // Set true to use image activeOverlay: false, // Set CSS color to display scrollUp active point, e.g #00FFFF zIndex: 2147483647 // Z-Index for the overlay }); }); /script>script type227bee229a7529d611148ae2-text/javascript> function responsiveMenu() { var x document.getElementById(menulist); if (x.className menulist) { x.className + open; } else { x.className menulist; }; var x document.getElementById(header); if (x.className page-header) { x.className + open; } else { x.className page-header; } } /script>script type227bee229a7529d611148ae2-text/javascript> (function(i,s,o,g,r,a,m){iGoogleAnalyticsObjectr;irir||function(){ (ir.qir.q||).push(arguments)},ir.l1*new Date();as.createElement(o), ms.getElementsByTagName(o)0;a.async1;a.srcg;m.parentNode.insertBefore(a,m) })(window,document,script,https://www.google-analytics.com/analytics.js,ga); ga(create, UA-1749595-6, auto); ga(send, pageview); /script>/head>body>header idheader classpage-header>h1 classproject-name>a classproject-name hrefhttps://fyrd.science>Fyrd/a>/h1>br>div classlogo>a hrefhttps://fyrd.science>img srchttps://fyrd.science/assets/logo/fyrd.png srcsethttps://fyrd.science/assets/logo/fyrd.svg altThe Fyrd Python Cluster logo — a Saxon shield reminiscent of those used in Fyrds />/a>/div>h2 classproject-tagline>A pythonic way to submit jobs to any cluster with dependency tracking./h2>nav idmenu>a hrefjavascript:void(0); onclickif (!window.__cfRLUnblockHandlers) return false; responsiveMenu() idmenu-button classbtn data-cf-modified-227bee229a7529d611148ae2->Menu/a>div classmenulist idmenulist>a hrefhttps://github.com/MikeDacre/fyrd classbtn>View on GitHub/a>a hrefhttps://fyrd.readthedocs.io classbtn>Read the Docs/a>a hrefhttps://github.com/MikeDacre/fyrd/archive/v0.6.2b1.tar.gz classbtn>Download Latest/a>a hrefhttps://pypi.python.org/pypi/fyrd classbtn>PyPI/a>a hrefhttps://fyrd.science/news classbtn>News/a>a hrefhttps://fyrd.science/releases classbtn>Releases/a>/div>/nav>/header>section classmain-content>p classbadges>a hrefhttps://buildkite.com/mikedacre/fyrd-cluster-tests>img srchttps://badge.buildkite.com/b6659b460caf5205919916c4e9d212c4e04d4301fa55a51180.svg?branchmaster altFyrd Buildkite build state for version master />/a>a hrefhttps://travis-ci.org/MikeDacre/fyrd>img srchttps://travis-ci.org/MikeDacre/fyrd.svg?branchmaster altFyrd Travis CI tests for the master version />/a>a hrefhttps://www.codacy.com/app/mike-dacre/fyrd?utm_sourcegithub.com&utm_mediumreferral&utm_contentMikeDacre/fyrd&utm_campaignBadge_Grade>img srchttps://api.codacy.com/project/badge/Grade/c163cff81a1941a18b2c5455901695a3 altCodacy Project Grade />/a>a hrefhttps://fyrd.readthedocs.io/en/latest>img srchttps://readthedocs.org/projects/fyrd/badge/?versionlatest altFyrd master Manual />/a>a hrefhttps://badge.fury.io/py/fyrd>img srchttps://badge.fury.io/py/fyrd.svg altPyPI version height18 />/a>img srchttps://img.shields.io/badge/python%20versions-2.7%203.3+-brightgreen.svg altimage />a hrefhttps://requires.io/github/MikeDacre/fyrd/requirements/?branchmaster>img srchttps://requires.io/github/MikeDacre/fyrd/requirements.svg?branchmaster altFyrd Version master Requirements Status />/a>/p>h3 idwhat-is-fyrd>What is em>Fyrd/em>?/h3>p>em>Fyrd/em> is a library for Python 2.7+/3.3+ that allows easy multiprocessing-style parallelization using a batch system back end. Right now it supports a hrefhttp://www.adaptivecomputing.com/products/open-source/torque/>torque/a> and a hrefhttps://slurm.schedmd.com/>slurm/a>, with a local, multiprocessing based, fallback. Because it is written in a modular way, however, any batch system can be added. To request your batch system be added, please email a href/cdn-cgi/l/email-protection class__cf_email__ data-cfemail3d4f584c48584e494e7d5b444f59134e5e5458535e58>email protected/a> or a hrefhttps://github.com/MikeDacre/fyrd/issues>submit an issue on the github page/a>./p>p>The local fallback mode is a miniature torque/slurm style job managing clone. It starts a server on your machine and that server will accept and run jobs in the same way that torque or slurm would. It allows dependency tracking to work, but does not implement partitions or allow the viewing of other users jobs. Note that fyrd’s local mode is not a fantastic substitute for multiprocessing, because it uses a file-submission model that is unnecessary for local jobs, this means that there is a slight (about 0.5 second) overhead for every job that is submitted./p>h3 ida-quick-example>A Quick Example/h3>p>Say you have some piece of code that you got from somewhere, lets call it code classhighlighter-rouge>really_slow_algorithm/code>, and you want to run it on 200 files and then work on then examine the results of code classhighlighter-rouge>STDOUT/code>, but you need to parse the results first to get just the first two columns. Here is how you would do that:/p>div classlanguage-python highlighter-rouge>div classhighlight>pre classhighlight>code>span classkn>import/span> span classnn>os/span>span classkn>import/span> span classnn>math/span>span classkn>import/span> span classnn>fyrd/span>span classn>SCRIPT/span> span classo>/span> span classs>really_slow_algorithm {} | awk {print $1, /span>span classse>\\/span>span classs>t, $2}/span>span classc># This is the same as running fyrd.Job(filter_results, (x,), profilesmall, time00:01:00).submit()/span>span classc># It is best used when dependency tracking is not needed and every function submission requires/span>span classc># the same resources/span>span classnd>@fyrd.jobify/span>span classp>(/span>span classn>profile/span>span classo>/span>span classs>small/span>span classp>,/span> span classn>time/span>span classo>/span>span classs>00:01:00/span>span classp>)/span>span classk>def/span> span classnf>filter_results/span>span classp>(/span>span classn>x/span>span classp>):/span> span classs>If first column contains tomato, return the log of the second column./span> span classn>output/span> span classo>/span> span classp>/span> span classk>for/span> span classn>i/span> span classow>in/span> span classn>x/span>span classo>./span>span classn>split/span>span classp>(/span>span classs>/span>span classse>\n/span>span classs>/span>span classp>):/span> span classn>name/span>span classp>,/span> span classn>data/span> span classo>/span> span classn>i/span>span classo>./span>span classn>split/span>span classp>(/span>span classs>/span>span classse>\t/span>span classs>/span>span classp>)/span> span classk>if/span> span classs>tomato/span> span classow>in/span> span classn>name/span>span classp>:/span> span classn>output/span>span classo>./span>span classn>append/span>span classp>(/span>span classn>math/span>span classo>./span>span classn>log10/span>span classp>(/span>span classnb>int/span>span classp>(/span>span classn>data/span>span classp>)))/span> span classk>return/span> span classn>output/span>span classn>script_jobs/span> span classo>/span> span classp>/span>span classk>for/span> span classn>i/span> span classow>in/span> span classn>os/span>span classo>./span>span classn>listdir/span>span classp>(/span>span classs>input_files/span>span classp>):/span> span classn>script_jobs/span>span classo>./span>span classn>append/span>span classp>(/span>span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>SCRIPT/span>span classo>./span>span classn>format/span>span classp>(/span>span classn>i/span>span classp>),/span> span classn>cores/span>span classo>/span>span classmi>1/span>span classp>,/span> span classn>mem/span>span classo>/span>span classs>10GB/span>span classp>,/span> span classn>time/span>span classo>/span>span classs>48:00:00/span>span classp>)/span>span classo>./span>span classn>submit/span>span classp>())/span>span classc># Wait for jobs to complete/span>span classn>fyrd/span>span classo>./span>span classn>wait/span>span classp>(/span>span classn>script_jobs/span>span classp>)/span> span classn>func_jobs/span> span classo>/span> span classp>/span>span classk>for/span> span classn>result/span> span classow>in/span> span classn>script_job/span>span classp>:/span> span classn>func_jobs/span>span classo>./span>span classn>append/span>span classp>(/span>span classn>filter_results/span>span classp>(/span>span classn>result/span>span classo>./span>span classn>stdout/span>span classp>))/span>span classn>filtered/span> span classo>/span> span classn>fyrd/span>span classo>./span>span classn>basic/span>span classo>./span>span classn>get/span>span classp>(/span>span classn>func_jobs/span>span classp>)/span>/code>/pre>/div>/div>h3 iddependency-tracking>Dependency tracking/h3>p>Fyrd implements a simple dependency tracking system. Most batch systems allow dependency tracking, but they require somewhat complex syntax and a prior knowledge of the job ID, something that you do not have when you are building a pipeline. em>Fyrd/em> therefore allows tracking of dependencies using a hrefhttps://fyrd.readthedocs.io/en/latest/api.html#fyrd.Job>Job objects/a>, as well as by job IDs, it you happen to have those:/p>div classlanguage-python highlighter-rouge>div classhighlight>pre classhighlight>code>span classn>job/span> span classo>/span> span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classs>really_slow_algorithm big_file/span>span classp>)/span>span classo>./span>span classn>submit/span>span classp>()/span>span classn>job/span>span classo>./span>span classn>wait/span>span classp>()/span>span classk>if/span> span classn>job/span>span classo>./span>span classn>completed/span>span classp>:/span> span classn>job2/span> span classo>/span> span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>completed_function/span>span classp>)/span>span classk>elif/span> span classn>job/span>span classo>./span>span classn>failed/span>span classp>:/span> span classn>job2/span> span classo>/span> span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>failed_function/span>span classp>)/span>span classn>job3/span> span classo>/span> span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>wrapup_function/span>span classp>,/span> span classn>depends/span>span classo>/span>span classn>job2/span>span classp>)/span>span classk>for/span> span classn>job/span> span classow>in/span> span classp>/span>span classn>job2/span>span classp>,/span> span classn>job3/span>span classp>:/span> span classn>job/span>span classo>./span>span classn>submit/span>span classp>()/span>span classn>result/span> span classo>/span> span classn>job3/span>span classo>./span>span classn>get/span>span classp>()/span>/code>/pre>/div>/div>h3 idkeyword-arguments-and-profiles>Keyword Arguments and Profiles/h3>p>Batch systems have many possible submission options, to allow these options to work the same irrespective of cluster syntax, they are all implemented as keyword arguments to the code classhighlighter-rouge>fyrd.job.Job/code> class. A complete list is available in the a hrefhttps://fyrd.readthedocs.io/en/latest/keywords.html>documentation/a>./p>p>Because there can be many possible arguments to the batch system, and because most jobs use the same few arguments, em>Fyrd/em> also has a a hrefhttps://fyrd.readthedocs.io/en/latest/basic_usage.html#profiles>profile system/a>, which allows common arguments to be bundled together. This system implements a code classhighlighter-rouge>DEFAULT/code> profile, which allows job submission with no arguments at all:/p>div classlanguage-python highlighter-rouge>div classhighlight>pre classhighlight>code>span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>my_function/span>span classp>)/span>span classo>./span>span classn>submit/span>span classp>()/span>span classo>./span>span classn>get/span>span classp>()/span>span classn>fyrd/span>span classo>./span>span classn>Job/span>span classp>(/span>span classn>long_function/span>span classp>,/span> span classn>profile/span>span classo>/span>span classs>long/span>span classp>)/span>span classo>./span>span classn>submit/span>span classp>()/span>span classo>./span>span classn>get/span>span classp>()/span>/code>/pre>/div>/div>h3 idinstallation>Installation/h3>div classlanguage-bash highlighter-rouge>div classhighlight>pre classhighlight>code>pip install fyrd/code>/pre>/div>/div>p>To get a specific version:/p>div classlanguage-bash highlighter-rouge>div classhighlight>pre classhighlight>code>pip install https://github.com/MikeDacre/fyrd/archive/0.6.2b1.tar.gz span classnt>--user/span>fyrd conf init/code>/pre>/div>/div>p>To get the latest stable version:/p>div classlanguage-bash highlighter-rouge>div classhighlight>pre classhighlight>code>pip install https://github.com/MikeDacre/fyrd/tarball/master span classnt>--user/span>fyrd conf init/code>/pre>/div>/div>p>To get the latest development version (still pretty stable):/p>div classlanguage-bash highlighter-rouge>div classhighlight>pre classhighlight>code>pip install https://github.com/MikeDacre/fyrd/tarball/dev span classnt>--user/span>fyrd conf init/code>/pre>/div>/div>p>The code classhighlighter-rouge>fyrd conf init/code> command initializes your environment interactively by asking questions about the local cluster system./p>h3 iddocumentation>Documentation/h3>p>Fyrd has very comprehensive documentation on a hrefhttps://fyrd.readthedocs.org>Read the Docs/a>, you should definitely read it. A good place to start is the a hrefhttps://fyrd.readthedocs.io/en/latest/basic_usage.html#>getting started documentation/a>./p>h3 idissues-and-contributing>Issues and Contributing/h3>p>If you have any trouble with this software add an issue on a hrefhttps://github.com/MikeDacre/fyrd/issues>the issues page/a>/p>p>I am always looking for help testing the software and implementing new keyword arguments. I would also very must like to add new batch system environments, but I need access to those clusters in order to test the new keyword arguments and implement queue parsing. If you would like to help with that or give me access to your cluster (for development of this package only), please email me a a href/cdn-cgi/l/email-protection class__cf_email__ data-cfemail88e5e1e3edc8eef1faeca6fbebe1ede6ebed>email protected/a>./p>h3 idwhy-the-name>Why the Name?/h3>p>I gave this project the name ‘Fyrd’ in honor of my grandmother, Hélène Sandolphen, who was a scholar of old English. It is the old Anglo-Saxon word for ‘army’, particularly an army of freemen, and this code gives you an army of workers on any machine or cluster so it seemed appropriate. The logo is an Anglo Saxon shield of the kind used by the Fyrds, with a graphic of a cluster superimposed on the top./p>p>The project used to be called “Python Cluster”, which is more descriptive but frankly boring. Also, about half a dozen other projects have almost the same name, so it made no sense to keep that name and put the project onto PyPI./p>footer classsite-footer>div classlogo>a hrefhttps://fyrd.science>img srchttps://fyrd.science/assets/logo/fyrd.png srcsethttps://fyrd.science/assets/logo/fyrd.svg altThe Fyrd Python Cluster logo — a Saxon shield reminiscent of those used in Fyrds height300px width300px />/a>/div>span classsite-footer-owner>a hrefhttps://fyrd.science>Fyrd/a> is maintained by a hrefhttps://www.michaeldacre.com>Mike Dacre/a>./span>span classsite-footer-info>This code is available at a hrefhttps://github.com/MikeDacre/fyrd>GitHub/a> and is released under a hrefhttps://github.com/MikeDacre/fyrd/blob/master/LICENSE>the MIT license/a>./span>br>span classsite-footer-credits>This page was generated by a hrefhttps://pages.github.com>GitHub Pages/a> using the a hrefhttps://github.com/pietromenna/jekyll-cayman-theme>Cayman theme/a> by a hrefhttps://github.com/jasonlong>Jason Long/a>./span>/footer>/section>script data-cfasyncfalse src/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js>/script>script src/cdn-cgi/scripts/7d0fa10a/cloudflare-static/rocket-loader.min.js data-cf-settings227bee229a7529d611148ae2-|49 defer>/script>/body>/html>
View on OTX
|
View on ThreatMiner
Please enable JavaScript to view the
comments powered by Disqus.
Data with thanks to
AlienVault OTX
,
VirusTotal
,
Malwr
and
others
. [
Sitemap
]