Tuesday, 3 March 2020

Prevent closed elements

There seems to be a lot of websites that offer methods of preventing closed elements. Not sure if this is of any relevance but if we use a zero space character then it does offer a good method of display for the rendered element

<element><xsl:value-of select="'&#65279;'"/></element>

which will render as

<element></element>

Tuesday, 23 October 2018

Dynamic geographic extent finder

This uses a power set to return all the applicable geographic extents based on a requested value. The code is based upon the functions detailed in David Cassel's blog

declare function search:ignore-word($word as element(cts:word-query)) as xs:boolean {
 $word/cts:text/lower-case(.) = $search:stopWords and not($word/@qtextpre = '"' and $word/@qtextpost = '"')
};


declare function search:print($index, $values, $selections, $connector) {
    fn:concat(fn:string-join($values[$selections], $connector))
};

declare function search:apply-on-power-set($seq as item()*, $function as xdmp:function, $data) {
    let $len := fn:count($seq)
    for $i in (1 to xs:int(math:pow(2, $len) - 1))
    let $targets :=
        for $bit in (1 to $len)
        let $shifted :=
            if ($bit > 1) then
                xs:int($i div (math:pow(2, $bit - 1)))
            else $i
        return
            if (math:fmod($shifted, 2) eq 1) then
                $bit
            else ()
    return
        xdmp:apply($function, $i, $seq, $targets, $data)
};

declare variable $search:extents as xs:string+ := ('E', 'W', 'S', 'N.I.', 'E.U.');

declare variable $search:extentcombinations as xs:string+ := search:apply-on-power-set(($search:extents), xdmp:function(xs:QName("search:print")), '+');

declare function search:extent-combinations($extents as xs:string+) as xs:string* {
distinct-values(
    for $e in $extents
    return
        if ($e = ('england', 'E')) then $search:extentcombinations[matches(., 'E$|E\+')]
       else if ($e = ('wales', 'W')) then $search:extentcombinations[matches(., 'W')]
       else if ($e = ('scotland', 'S')) then $search:extentcombinations[matches(., 'S')]
       else if ($e = ('ni', 'N.I.')) then $search:extentcombinations[matches(., 'N\.I\.')]
       else if ($e = ('eu', 'E.U.')) then $search:extentcombinations[matches(., 'E\.U\.')]
       else ()
      )
};

Tuesday, 2 October 2018

Bluetooth not loading in Windows 10

Issue

When starting windows 10 the bluetooth driver fails to load and refuses to start manually from the services

Resolution

click Windows Icon + X then selct Device Manage. Delete the Bluetooth driver and delete - windows will replace and resolve the issue

Thursday, 2 August 2018

Returning xml data from a json file with XSLT 3

XSLT is enabled with functions to support json. This can be used with a simple call to retrieve and convert a json file to xml

 <xsl:sequence select="json-to-xml(unparsed-text($my-uri))"/>

Friday, 27 July 2018

xquery return a regex matched string

Useful function to return the string matched by a regex pattern - note this will only work for a single match in the string

 declare function local:get-string-match($string as xs:string?, $regex as xs:string)  as xs:string? {
 if (matches($string,$regex)) then
  let $length := string-length($string) - string-length(replace($string, $regex,''))
  let $start := string-length(tokenize($string, $regex)[1]) + 1
  return substring($string, $start, $length)
 else ()
 } ;


Wednesday, 6 June 2018

Resolve Curls double header when requesting from a digest authentication source in an xslt pipeline

When using curl to make a request against a source that uses digest authentication the response will contain a double header, the first is a 401 response followed by the actual response of the request. This is because using the --digest flag with curl, the request will firstly return a 401 to get the nonce etc to be able to do the second request that would get the real data.

A sample request would be something like:

curl http:www.mysite.com --digest -u user:pw --dump-header headers.txt

An example of the response from such a request is shown below:

HTTP/1.1 401 Unauthorized
Server: MarkLogic
WWW-Authenticate: Digest realm="public", qop="auth", nonce="563c5b25a3dbad7b6aa54cdc7dc0d294", opaque="56e3687fefa06c25"
Content-Type: text/html; charset=utf-8
Content-Length: 209
Connection: Keep-Alive
Keep-Alive: timeout=5

HTTP/1.1 200 OK
Server: MarkLogic
Content-Type: application/xml; charset=UTF-8
Content-Length: 175
Connection: Keep-Alive
Keep-Alive: timeout=5

When this is a part of an xml/xslt workflow then which makes use of the header file that is returned then the following code will resolve to the correct header with the following considerations:

  • This is merely a code snippet that requires to be employed within an xslt stylesheet
  • The code makes use of functions from the http://www.xsltfunctions.com library (xmlns:functx="http://www.functx.com")
  • The $actualHeader variable is the header that is required

<xsl:variable name="unparsedText"  select="unparsed-text(resolve-uri('headers.txt', resolve-uri('mydir', static-base-uri())))"/>
<xsl:variable name="actualHeader"  
  select="if (tokenize($unparsedText, 'HTTP')[3]) then 
       substring($unparsedText, functx:index-of-string-last($unparsedText,'HTTP')) 
   else $unparsedText"/>

<xsl:function name="functx:index-of-string-last" as="xs:integer?">
  <xsl:param name="arg" as="xs:string?"/>
  <xsl:param name="substring" as="xs:string"/>

  <xsl:sequence select="
  functx:index-of-string($arg, $substring)[last()]
 "/>

</xsl:function>

<xsl:function name="functx:index-of-string" as="xs:integer*">
  
  

  <xsl:sequence select="
  if (contains($arg, $substring))
  then (string-length(substring-before($arg, $substring))+1,
        for $other in
           functx:index-of-string(substring-after($arg, $substring),
                               $substring)
        return
          $other +
          string-length(substring-before($arg, $substring)) +
          string-length($substring))
  else ()
 "/>

</xsl:function>


Thursday, 24 May 2018

Remove illegal XML characters

A recent issue where some parsed text generated from PDFbox required to be transformed using XSLT highlighted issues with illegal characters in the content. This was part of an ant task and therefore a simple replaceregex solved the issue. The matter was complicated by characters that had not correctly translated to UTF-8, notably the greek π character. Therefore a couple of regexes were required to resolve the issue. The first matches all characters that are not part of the XML specification, the next is specific to the the π character

<target name="remove-illegal">
 <replaceregexp match="[^\x09\x0A\x0D\x20-\uD7FF\uE000-\uFFFD\u10000-u10FFFF]" replace="" flags="g" encoding="utf-8">
  <fileset dir="PDFbox-text" includes="*.txt **/*.txt"/>
 </replaceregexp>
 <replaceregexp match="[\\xcf]" replace="" flags="gs" encoding="utf-8">
  <fileset dir="PDFbox-text" includes="*.txt **/*.txt"/>
 </replaceregexp>
</target>


Thursday, 1 March 2018

XSLT: Format number as an alphabetic value

Here is a useful pair of functions that will format a numeric value as an alphabetic value. Ideal for generating alphabetic list numbers from integers. This is not limited to the 26 alphabetic characters and will recurse through the alphabet for each multiple of 26 such that the list will grow from a-z, aa-zz, aaa-zzz etc

xmlns:xslttricks="http://http://xslt-tricks.blogspot.co.uk/functions"


 <xsl:function name="xslttricks:format-number-as-alpha">
  <xsl:param name="number" as="xs:decimal"/> 
  <xsl:value-of select="leg:format-number-as-alpha($number, ())"/>
 </xsl:function>

 <xsl:function name="xslttricks:format-number-as-alpha">
  <xsl:param name="number" as="xs:decimal"/>
  <xsl:param name="case" as="xs:string?"/>
  <xsl:variable name="int" select="xs:integer(round($number))"/>
  <xsl:variable name="mod" select="$int mod 26"/>
  <xsl:variable name="times" select="xs:integer(floor($int div 26) +1)"/>
  <xsl:variable name="alpha" select="('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z')"/>
  <xsl:variable name="numberstring" select="string-join((for $n in 1 to $times return $alpha[$mod]), '')"/>
  <xsl:value-of select="if ($case = ('upper', 'uppercase')) then upper-case($numberstring) else $numberstring"/>
 </xsl:function>


Saturday, 28 October 2017

XSLT functions for distance conversions

Here are two simple XSLT functions for converting between miles and km

 <xsl:function name="fn:miles-to-km">
  <xsl:param name="miles" as="xs:decimal"/> 
  <xsl:sequence select="format-number($miles * 1.609347, '0.0')"/>
 </xsl:function>
 
 <xsl:function name="fn:km-to-miles">
  <xsl:param name="km" as="xs:decimal"/> 
  <xsl:sequence select="format-number($km * 0.621371, '0.0')"/>
 </xsl:function>

Wednesday, 4 October 2017

XSLT to convert Excel (xls) to html

Issue

Require html from an xls (Excel) spreadsheet

Resolution

Quick and dirty XSLT transformation

Xquery

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
     xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" 
    xmlns="http://www.w3.org/1999/xhtml"
 exclude-result-prefixes="ss"
         version="2.0">
 <xsl:output method="html"/>

  <xsl:template match="/">
    <html>
       <head>
        <style>
         <xsl:apply-templates select="ss:Workbook/ss:Styles/ss:Style"/>
        </style>
       </head>
     <body>
      <div id="layout2">
       <xsl:apply-templates select="ss:Workbook/ss:Worksheet/ss:Table"/>
      </div>
     </body>
    </html>
  </xsl:template>
 
  <xsl:template match="ss:Style">
   <xsl:text>#</xsl:text>
   <xsl:value-of select="@ss:ID"/>
   <xsl:text> {</xsl:text>
   <xsl:if test="ss:Alignment/@ss:Horizontal">
   <xsl:text>text-align: </xsl:text>
   <xsl:value-of select="ss:Alignment/@ss:Horizontal"/>
   <xsl:text>;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Alignment/@ss:Vertical">
   <xsl:text>vertical-align: </xsl:text>
   <xsl:value-of select="ss:Alignment/@ss:Vertical"/>
   <xsl:text>;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Alignment/@ss:Vertical">
   <xsl:text>vertical-align: </xsl:text>
   <xsl:value-of select="ss:Alignment/@ss:Vertical"/>
   <xsl:text>;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Font/@ss:Bold=1">
   <xsl:text>font-weight: bold;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Font/@ss:Color">
   <xsl:text>color: </xsl:text>
   <xsl:value-of select="ss:Font/@ss:Color"/>
   <xsl:text>;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Font/@ss:StrikeThrough=1">
   <xsl:text>text-decoration: line-through;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Font/@ss:Underline='Single'">
   <xsl:text>text-decoration: underline;</xsl:text>
  </xsl:if>
  <xsl:if test="ss:Interior/@ss:Color">
   <xsl:text>background-color: </xsl:text>
   <xsl:value-of select="ss:Interior/@ss:Color"/>
   <xsl:text>;</xsl:text>
  </xsl:if>
  <xsl:for-each select="ss:Borders/ss:Border">
   <xsl:if test="@ss:Color">
    <xsl:text>border-</xsl:text>
    <xsl:value-of select="@ss:Position"/>
    <xsl:text>-color:</xsl:text>
    <xsl:value-of select="@ss:Color"/>
    <xsl:text>;</xsl:text>
   </xsl:if>
   <xsl:if test="@ss:Weight">
      <xsl:text> border-</xsl:text>
      <xsl:value-of select="@ss:Position"/>
      <xsl:text>-width:</xsl:text>
      <xsl:value-of select="@ss:Weight"/>
      <xsl:text>px;</xsl:text>
   </xsl:if>
   <xsl:if test="@ss:LineStyle">
    <xsl:text>border-</xsl:text><xsl:value-of select="@ss:Position"/><xsl:text>-style:</xsl:text>
    <xsl:value-of select="@ss:LineStyle"/>
    <xsl:text>;</xsl:text>
   </xsl:if>
  </xsl:for-each>
  <xsl:text>}</xsl:text>
  <xsl:text>.scrolltable {overflow: scroll;}</xsl:text>
 </xsl:template> 
 
 <xsl:template match="ss:Table">
  <div class="scrolltable">
   <table>
    <xsl:apply-templates select="ss:Row"/>
   </table>
  </div>
 </xsl:template>
 
 <xsl:template match="ss:Row">
  <tr>
   <xsl:apply-templates select="ss:Cell"/>
  </tr>
 </xsl:template>
 
 <xsl:template match="ss:Row[@ss:Hidden = '1']" priority="10"/> 
 
 <xsl:template match="ss:Cell">
  <td>
   <xsl:attribute name="id">
    <xsl:value-of select="@ss:StyleID"/>
   </xsl:attribute>
   <xsl:if test="@ss:MergeAcross">
    <xsl:attribute name="colspan">
     <xsl:value-of select="@ss:MergeAcross+1"/>
    </xsl:attribute>
   </xsl:if>
   <xsl:value-of select="ss:Data"/>
  </td>
 </xsl:template>
 
</xsl:stylesheet>

Saturday, 6 August 2016

xslt function substring-between()

Issue

Need to return a substring between two strings

Resolution

xslt function

code

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" 
xmlns:gm='http://www.griffmonster.org/xslt2/functions' 
version="2.0" exclude-result-prefixes="gm">

<xsl:function name="gm:substring-between">
 <xsl:param name="string"/>
 <xsl:param name="start-string"/>
 <xsl:param name="end-string"/>
 <xsl:sequence select="substring-before(substring-after($string, $start-string),$end-string)"/>
</xsl:function>



</xsl:stylesheet>

Monday, 4 January 2016

Determine the position and content of an item in a string sequence

Issue

We have a sequence of strings and for each item we need the position of the item within the sequence

Resolution

use the following xpath expression

Xquery

let
  $items := tokenize('xslt-tricks.blogspot.co.uk/search/label/xquery','/')
return
  for $i in 1 to count($items) 
  return (
   $i, (: position  :)
   $items[$i]  (: item content :)
  )

Friday, 20 November 2015

Xquery compare two nodesets

Issue

Compare two nodesets for equality using xquery

Resolution

Use a recursive function to runa deep-equal() function across all nodes

Xquery

declare function utils:deep-equal($node1 as element(), $node2 as element()) as xs:boolean {
    let $equal as xs:boolean := deep-equal($node1, $node2)
    return (
      if (not($equal) or empty($node1/*)) then $equal
      else  (
        every $de in (
            for $n in $node1/*
            let $position := count($n/preceding-sibling::*)+1
            return utils:deep-equal($n, $node2/*[$position])
            )
        satisfies $de = true()
        )
    )
};

Thursday, 22 October 2015

MarkLogic fails to install with 'Rolling Back Action' Message

In a recent MarkLogic upgrade on Windows Server 2008 the following issue was encountered:

The uninstall of the previous version successfully completed. The upgrade installer was initiated and it went thorugh the install process. At the very end it posted a message 'Rolling Back Action' and promprly reversed the installation.

The issue was caused by the inital uninstall not removing the MarkLogic service. Check this in the task manager services panel. If the MArkLogic service is there then run the command line with the following command:

sc delete MarkLogic

If this fails to remove the service but returns a message along the lines of 'this has been marked for deletion', then reboot the machine and the service should get removed. The installation can then be run successfully

Thursday, 27 August 2015

Monitor log files in real time

use the tail unix command for which there are windows equivalents

Links

http://tailforwin32.sourceforge.net/index.php

http://unxutils.sourceforge.net

Example command line

@D:\Logs\tail -f D:\Squid\var\logs\access.log

Thursday, 6 August 2015

Return all variables and functions from an xquery module using xslt

Issue

Require an XML listing of all variables and functions used in an xquery module

Resolution

The following XSLT is a quick and dirty method to extract all the variables and functions from an xquery module into an XML representation which can then be transformed to the required format

The transformation is XSLT2 and can be invoked by use of saxon using the following command line, where thisXSLT.xsl is the code below:

java -jar saxon.jar -it:main -xsl:thisXSLT.xsl -o:result.xml "csvFile=myfile.csv"

XSLT

<xsl:stylesheet 
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:xs="http://www.w3.org/2001/XMLSchema"
    version="2.0">

 
 <xsl:param name="xquery" as="xs:string" >utils.xq</xsl:param>
 
 <xsl:output indent="yes"/>

 <xsl:variable name="input-text" 
              as="xs:string" 
              select="unparsed-text($xquery, 'utf-8')"/>

 <xsl:template match="/" name="main">
  <xquery>
  <variables>
   <xsl:analyze-string select="$input-text" 
        regex="(declare)\s+(variable)\s+\$((\c)+:(\c)+)(\s+as\s+.+)?\s+(:=)"> 
     <xsl:matching-substring>
      <xsl:variable name="arguments" as="xs:string*" select="tokenize(normalize-space(regex-group(5)),',')"/>
        <variable>
         <name datatype="{if (matches(regex-group(6), 'as ')) then substring-after(regex-group(6), 'as ') else 'undefined'}"><xsl:value-of select="normalize-space(regex-group(3))"/></name>
         <!--<xsl:value-of select="."/>-->
        </variable>
     </xsl:matching-substring>
     <xsl:non-matching-substring>
      <xsl:message>
      Non-matching line "<xsl:value-of select="."/>"
      </xsl:message>
     </xsl:non-matching-substring>
    </xsl:analyze-string>
  </variables>
  <functions>
   <xsl:variable name="regexFnStart">\{</xsl:variable>
   <xsl:analyze-string select="$input-text" 
       regex="(declare)\s*(function)\s+((\c)+)\(((.)*)\)(\s+as\s+.+)?\s+({$regexFnStart})"> 
    <xsl:matching-substring>
     <xsl:variable name="arguments" as="xs:string*" select="tokenize(normalize-space(regex-group(5)),',')"/>
       <function>
        <name datatype="{if (matches(regex-group(7), 'as ')) then substring-after(regex-group(7), 'as ') else 'undefined'}"><xsl:value-of select="normalize-space(regex-group(3))"/></name>
        <xsl:for-each select="$arguments">
         <xsl:variable name="datatype" as="xs:string?" select="if (matches(., ' as ')) then substring-after(., ' as ') else ''"/>
         <argument datatype="{if (matches($datatype,'\)$') and not(matches($datatype,'\(\)$'))) then replace($datatype, '\)$' ,'') else $datatype}">
          <xsl:value-of select="normalize-space(if (matches(., ' as ')) then substring-before(., ' as ') else .)"/>
         </argument>
        </xsl:for-each>
       <!--<xsl:value-of select="."/>--></function>
    </xsl:matching-substring>
    <xsl:non-matching-substring>
     <xsl:message>
     Non-matching line "<xsl:value-of select="."/>"
     </xsl:message>
    </xsl:non-matching-substring>
   </xsl:analyze-string>
  </functions>
  </xquery>
 </xsl:template>

Friday, 17 April 2015

xml validator

A quick an easy command line XML validator can be found at:

https://github.com/amouat/xsd-validator

http://www.adrianmouat.com/bit-bucket/2013/11/xml-schema-validation/

This is a simple java wrapper for xerces - what more could one want for simple validation

Even better why not set the path in the path environment variable and then call it from anywhere. sorted

Thursday, 26 March 2015

XSLT group by postion

Issue

Need to create individual files each with n nodes from the master file

Resolution

There is a simple resolution to this although it is specific to XSLT 2. This uses the for-each-group routine and the idiv operator. idiv is an integer division which is equivalent to (x div y) cast as xs:integer

Consider a nodeset held as a variable $nodes and a group size of nodes defined by the integer variable $groupsize, then using the following routine will split the nodeset into distinct files with $groupsize nodes to each file

XSLT

<xsl:for-each-group select="$nodes" 
    group-adjacent="(position() - 1) idiv $groupsize">
 
    <xsl:result-document href="{concat('resultdoc', format-number(position(),'000'), '.xml')}">
        <result>
            <xsl:sequence select="current-group()"/>
        </result>
    </xsl:result-document>
</xsl:for-each-group>

Thursday, 12 March 2015

netstat switches

The following switches can be used with the ipconfig command line utility:

SwitchEffect
-a Displays all connections and listening ports
-bDisplays the executable involved in creating each connection or listening port
-eDisplays Ethernet statistics
-fDDisplays Fully Qualified Domain Names for foreign addresses. (In Windows Vista/7 only)
-nDisplays addresses and port numbers in numerical form
-o Displays the owning process ID associated with each connection
-p protocol Shows connections for the protocol specified by proto; proto may be any of: TCP, UDP, TCPv6, or UDPv6.
-rDisplays the routing table
-s Displays per-protocol statistics
-tDDisplays the current connection offload state, (Windows Vista/7)
-v When used in conjunction with -b, will display sequence of components involved in creating the connection or listening port for all executables. (Windows XP SP2, SP3)
[interval] An integer used to display results multiple times with specified number of seconds between displays. Continues until stopped by command ctrl+c. Default setting is to display once

Common usage netsta -a -n -o

Use the pid to cross reference with the process running in Task Manager

Xquery: for $x at $position in $seq expression

A useful expression available to xquery is the "for x at $position in $seq expression" (for at in expression)


for $x at $position in $seq
 return
  if ($position = 1) then
   (: do something :)
  else if ($position = 2) then
   (: do something else :)
  else
   (: default :)